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Preface 


For the thirty-seventh time, the Research and Theory Division of the Association for Educational 
Communications and Technology (AECT) is sponsoring the publication of these Proceedings. Papers 
published in this volume were presented at the annual AECT Convention in Indianapolis, IN. A limited 
quantity of these Proceedings were printed and sold in both hardcopy and electronic versions. Volumes 1 
and 2 are available through the Educational Resources Clearinghouse (ERIC) System. Proceedings volumes 
are available to members at AECT.ORG. Proceedings copies are also available at: 

http://www.tresvstems.com/proceedings/ 

The Proceedings of AECT’s Convention are published in two volumes. Volume #1 contains papers dealing 
primarily with research and development topics. Papers dealing with the practice of instructional 
technology including instruction and training issues are contained in Volume #2. This year, both volumes 
are included in one document. 

REFEREEING PROCESS: Papers selected for presentation at the AECT Convention and included in these 
Proceedings were subjected to a reviewing process. All references to authorship were removed from 
proposals before they were submitted to referees for review. Approximately sixty percent of the 
manuscripts submitted for consideration were selected for presentation at the convention and for 
publication in these Proceedings. The papers contained in this document represent some of the most current 
thinking in educational communications and technology. 


Michael R. Simonson 
Editor 
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Student' Opinions and Perceptions about a 
Gamifled Online Course: A Qualitative Study 

Tugce Aldemir 

Pennsylvania State University 

Goknur Kaplan Akilli 

Middle East Technical University 

Abstract 

This study proposes the pilot results of a master thesis and reports on a practice of instructional design and 
development utilizing an innovative instructional design, namely, gamification. The study is designed as qualitative 
research that aims to examine students’ opinions and perceptions about an online course which was redesigned using 
a gamification theoretical framework entitled D6. The course was offered to the Department of English Language 
Teaching (ELT) students in the 2014-2015 Fall Semester and lasted for 14 weeks. Out of 81 ELT students, 17 were 
selected through convenient sampling to conduct semi-structured interviews. The collected data was analyzed 
through qualitative methods by utilizing open coding. The findings show that the perceptions and the opinions of the 
students about the gamified online course can be classified under four main themes: (a) General opinions and 
perceptions toward gamification, (b) Learner related themes, (c) Instructor related themes and (d) Gamified course 
related themes. 


Introduction 

Digital revolution has brought upon us the Digital Age, where everything in life is digitized. Advances in 
technology have made it possible to implement changes in all kinds of fields. These noticeable changes in the 
framework of our daily lives generate numerous kinds of needs and demands. Since the last decade of the 20 th 
century, a generation of ‘digital natives’, whose lives have been entangled in these technologies, has emerged with 
these new needs (Prensky, 2001). Just as many fields technological advances of the Digital Revolution has 
influenced, education would also have to accede and incorporate the new needs imposed by the generation of digital 
natives. In order to create a learning environment for this generation, which is also referred to as Generation Z 
(Levickaite, 2010; Tulgan, 2015), it is imperative to understand their characteristics and needs. The irony here is that 
the increasingly wide usage of technology in schools and formal education has had a disengaging effect on 
Generation Z (Mcgonigal, 2011). Possibly due to being born and raised in an interactive online environment, which 
has become a part of their lives, Generation Z prefers to learn anywhere except at the school, as a traditional school 
environment may easily bore them (Prensky, 2005). Engaging activities such as video games offer learners an 
environment where they can have fun while acquiring skills and knowledge as by-product (Gee, 2005). Hence the 
Generation Z has enjoyed these types of interactive and fun activities since birth; they seem to suffer from 
engagement problems in traditional school settings (McGonigal, 2011). 

Edutainment, educational games, have emerged as a result of the aforementioned engagement problems of 
Generation Z (Aslan and Balci, 2005). Serious games provide great benefits for learning environments (Lieberman, 
2006). However, they exist in virtual reality and are separated from real life, have different notions of space and 
time and are bounded within a magical circle (Huizinga, 1955). They are mainly designed to teach a particular 
content, consequently, rather than designing games for numerous kinds of activities, it would be more proficient to 
extract elements that make games fun and implement them with pertinent content intended to be taught. This process 
of applying game elements in a non-game context is called gamification (Deterding et al. 2011). Gamification has 
become a popular word in education, the main reason for this has been to help solve motivation and engagement 
problems of learners (Kapp, 2012). Proponents of the gamification agree that it has a great potential (Kapp, 2012); 
while its opponents list several design and perception related issues (Groh, 2012; Bogost, 2011). Gamification is 
mainly used in marketing field to attract and engage customers, and to create loyalty to a particular product. 
Foursquare and Nike+ are a couple successful examples who have proved gamifications potential in marketing. This 
frequently used concept, however, has not been studied exclusively from an educational perspective, so as to fully 
take advantage in education. Thus, this study aims to explain and explore the application of gamification elements to 
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an instructional setting and to gain a deeper understanding of gamification within education. For that purpose, this 
study sought to put forward the opinions and the perceptions of the students about an online course that was 
redesigned following the D6 gamification model, proposed by Werbach and Hunter (2012) for marketing for the the 
pilot study of a master thesis. 

In order to better shed light on the structure of the study, in the second part, fundamental literature about 
the issue is elaborated. Later, the methods of the study with respect to the description of the redesigned course: 
gamification of the course; participants and sampling; data collection and analysis methods are presented. Finally, 
the findings, the conclusion drawn from the findings and suggestions for a gamified online course are proposed. 


Literature Review 

Technological developments in information and knowledge do not recognize any boundaries any more as it 
has render it possible for people to reach knowledge without any time and space limit. In other words we can no 
longer consider the ability to obtain knowledge from a variety of sources as a valued skill. Thanks to technological 
advancement all the hitherto barriers between the people and institutions have been smashed and led to 
contemporary forms of networks between them (Beetham and Sharpe, 2013). In making the claim that technologies 
do not really enhance human capacity to learn better Beetham and Sharpe (2013) say that chalk and papyrus were 
considered to be technologies in the past. Vygotsky’s (1978) contention that social environment is quite vital in 
learning as a developmental process has to be taken quite seriously by the virtue of the fact that social environment 
does contain ‘the prior existence of complex cognitive structures’. These cognitive structures, which are vital 
constituent elements of the learning environment (culture) and have been internalized by the learner, may include 
variety of tools including physical materials, linguistic tools and resources such as technologies. The new paradigm 
built on this understanding operates with a new meaning of the term pedagogy in the digital age which gives high 
premium to the changing environment/context brought about by the changing technologies. The alterations made in 
the meaning of the term pedagogy is not interested in the question of advantages or disadvantages of technological 
changes, instead it is concerned with the newly formed collective intelligence (Segaran, 2007). The significant thing 
about the new collective intelligence is the fact that the learners can either contribute to or withdraw any knowledge 
from it. The collective intelligence in which the learners have been born and raised has enabled them to obtain new 
skills and knowledge with which they can meet the requirements of their daily lives, work and education. In recent 
decades the needs and the characteristics of the Generation Z as the product of such an environment has become a 
significant area of research and analysis. 

One of the foremost characteristics of the Generation Z, defined by Levickaite (2010) as the people bom in 
the mid-1990s and brought up in the 2000s, is multitasking as they attempt to carry out several activities at the same 
time instead of focusing on a specific task. For instance they can combine watching a video with looking at 
someone’s profile on Facebook and reading a news item in a paper. The vast world of few clicks away user- 
generated information that is the product of the recent technological and information developments has enabled the 
Generation Z to continuously and instantly change focus (Levickaite, 2010). 

It is rather strange that the wide use of technology in formal education and schools has had a disengaging 
impact on Generation Z (Mcgonigal, 2011). This is a rather paradoxical situation in that the learners prefer to learn 
in any place except the school. The main reason for this is the comparison made by the generation Z between the 
traditional school works with those of engaging activities made possible by the interactive online environment in 
which they have been born. 

It is clear that school work bores and frustrates Generation Z (Prensky, 2005), yet on the other hand they 
find highly engaging video games and the like both highly entertaining and at the same providing them the 
opportunities to learn new skills and knowledge. (Gee, 2005). With such qualities it is not surprising that games 
have been integrated into educational context. The history of using games in education can be traced back to the 
times of the emergence of games. However, only recently there has been the realization that games might be used in 
education with positive impacts. It is this late recognition that is behind the infancy of research in this area. The 
following brief quotation from Lieberman (2006, p.380) may be useful as an introductory statement: “All games are 
educational games. The question is: What are they teaching?” Reigeluth and Squire (1998) contention that games 
have the potential of motivating learners) may be the first answer to this question Similarly, Prensky (2001) puts an 
emphasis on the potential that games may provide a learner-centered, more entertaining and more captivating 
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learning experience. Critical thinking and problem solving skills are the other features that are positively associated 
with playing games (McFarlane, et al.). However despite the nicety of the positive assertions raised in the current 
literature their validity is highly questionable with respect to the scarcity of empirical studies, and the limited and 
contradictory nature of the evidence provided by them. Furthermore limited availability and expensiveness of the 
educational games (Cruickshank & Telfer, 1980), having the same boring content as with the traditional education 
(Prensky, 2011) and focusing on extrinsic motivation than intrinsic motivation (Egenfeldt-Nielsen, 2007) are 
considered to be some of the main problems faced by them 

Notwithstanding the above problems there are significant amount of interest in educational games in both 
the academic circles and the game-design sector due to the contention that the ability of games to offer a highly 
beneficial learning context (McFarlane, et al., 2002). Despite this positive note it must be noticed that it is not 
possible to design games for every single school activity or daily unpleasant activities. This realization necessitates 
the need to break the magical circle of the games. The term gamification emerged as a result of these concerns, yet 
not with the idea of entirely replacing serious game trend but to run parallel to it. Integrating game elements into 
non-game contexts was introduced in order to make the activities enjoyable and fun (Deterding, et al., 2011). 

Originating from the digital media industry in 2008 and becaming widely known in the second half of 2010 
(Deterding, et al., 2011) the term gamification is based on notion of motivating and engaging people in a game and 
do real-life activities using the game elements (Zichermann and Cunningham, 2011). “Gamification is the use of 
game design elements in non-game contexts” (Deterding, et al. 2011, p.2). As such its great potential in engaging, 
motivating, activating targeted behaviors and building loyalty to the gamified experience has to be emphasized 
(Deterding, et al., 2011). The potential of gamification to be used widely in everyday life also has a potential 
(Helgason, 2010) to help breaking the boundaries games have, and thus making them more omnipresent. Education 
is the most significant field in which gamification provides a huge potential (McGonagall, 201 1) by virtue of the fact 
that it can increase learners motivation towards the school activities and to obtain certain practical skills 
(Dominguez, et al., 2013). Having said this it must be said that gamification concept has its critics as well. One thing 
they raise is its nature of just being composed of game elements at the expense of critical game design that would 
motivate and engage people (Bogost, 2011). The other criticism directed at gamification is its tendency to turn 
intrinsic motivation into extrinsic motivation through a process of h integrating points, rewards and badges into 
almost everything which may lead to over-justification. (Lepper, et al., 1973). It seems that the opponents of the 
gamification see the design issues as the most problematic area. The main issue they raise is the question of the 
conglomeration of the elements in games. For them conglomeration of game elements does not necessarily mean a 
successful gamification application. What is important is the way in which these elements are combined. Such 
combination should pay serious attention to how the design fits perfectly with the goals of the system and motivates 
and engages players (Mcgonigal, 2011). In order to address this issue Werbach and Hunter (2012) has attempted to 
build a gamification design framework to be used in businesses. 

Werbach and Hunter’s (2012) D6 gamification model specifically produced to be used in marketing 
iscomposed of six basic design steps, each of which starts with the letter D: Define business objectives; delineate the 
target behaviors; describe your players; devise activity loops; don’t forget the fun; deploy the appropriate tools. As it 
was one of the few available design models for gamification Werbach and Hunter’s D gamification model was used 
in this study to redesign the course. 


Method 

This study aims to explore a gamified online course, and examine the opinions and perceptions of the 
students toward this course. On the basis of this purpose, this study focuses on the following research questions: 

•S What are the opinions of the students about the gamified online course? 

•S What are the perceptions of the students about the gamified online course? 

•S How can the gamified course be changed to better meet the needs of the students? 

Qualitative research was selected as the research approach since the goal of the study is to obtain insight 
and deep understanding about gamification. Creswell (2011, p.3) defines qualitative study as ‘an approach for 
exploring and understanding the meaning individuals or groups ascribe to a social or human problem’, which is 
appropriate for the goal of this study. 
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Course Description 


The case selected for the study is an undergraduate course titled CEIT 319-Instructional Technology and 
Material Development offered by the Department of Computer Education and Instructional Technology to other 
departments in the Faculty of Education. There is no prerequisite to take this course; as such it is open to variety of 
students at different stages in their education process: i.e. sophomores, juniors. The main objective of the course is 
teaching how to integrate technology in education process and how to prepare pertinent materials using technology. 

Gamifieation of the Course 

D6 gamifieation design framework by Werbach and Hunter (2012) was followed in gamifying the online 
course. This framework was selected on account of it having parallel steps with instructional design models such as 
ADDIE, hence it being easier to adapt to educational context. 

The first step, defining business objectives, is adapted as defining learning objectives. The goals and 
objectives of the course were designed with the purpose of setting it as a regular course. Ten presentations prepared 
for the course content along with supplementary materials were uploaded to a Web 2.0 tool apropos to outline target 
behaviors in line with the defined goals and objectives. The presentations are created to include some randomly 
placed challenges (quests) for students. Students are expected to read the content and keep up with the challenges. 
Points are awarded based on their performances. Trying to keep students in the flow throughout this process was a 
challenging problem. Therefore, to motivate students, three factors of Fogg behavior model (2009) were applied. 
Each student was sent an acceptance letter written in a fantastical narrative (based on Harry Potter genre) at the 
beginning of the semester to inform them about the structure of the course as well as to motivate them for the next 
levels within the course. The first quest was set on an easy level to increase student motivation. Difficulty level of 
the following quests gradually increased. As a trigger for each level, e-mails with detailed information about the 
quests and the possible awards for completing the quests were delivered to students. Four particular groups (houses) 
were established to describe the different type of players; student filled out a ‘player type test’ to be grouped with 
other students possessing similar characteristics. Progress and engagement loops were defined for devising the 
activity loops. Students were informed to read the content and solve challenges for engagement loop. In return, they 
received individual feedback and points based on their performances, which corresponded to different levels of 
badges. Collected points, along with the performance of the student’s house were used to create leaderboards to 
show the top 10 students for each week. A progression loop illustrating the completion of the increasingly difficult 
levels within the quests was designed to demonstrate students’ progress within the course. Furthermore, an online 
map was created for students to track their progress. Lazarro’s four keys model (2004) was utilized to incorporate 
‘fun’. Different types of random quests were given to the students to test their understanding of the content and to 
enhance their reflective skills. Leaderboards and badges were designed to give students the feeling of win state. For 
altered states, some small surprises were placed in each week’s content to give students short mental breaks while 
studying. All of the badges were designed in cooperation with the fantastical narrative. Students were required to 
help each other on the online system with point awards to account for people factor. To collectively rise their house 
to the top, each student was responsible for one another as well. Various Web 2.0 LMS tools: Schoology, Moodle, 
Edmodo were examined as possible deployment options for the appropriate tool. Although future studies may point 
to a better interface to use in an online gamified course; of the provided tools, being the one tool that satisfied all the 
design needs for the course, Edmodo was deemed most feasible. 

Participants and Sampling 

A two-stage sampling was utilized in the study: first, course-level sampling was done to select the course 
for gamifieation and then, participant-level sampling was done to select the participants for data collection. For the 
course, convenient sampling was done since the instructor of the course was willing to cooperate in gamifying her 
course. For participants, another convenient sampling was done, and information rich cases were selected from the 
participants who volunteered to participate in semi-structured interviews. 

The information gathered from the demographic questionnaire administered to the participants at the 
beginning of the semester reveals the following. There were 6 freshman (%7.4), 51 sophomore (%63), 23 junior 
(%28.4) student participants (/?=80). Participants’ GPA ranged between 1.73 and 4.00 (M=3.12, SD= 0.53, n=72) and 
their ages ranged between 18 and 22 (M=19.38, SD— 0.6, N=Sl). Table 1 shows the distribution of the participants by 
gender and whether they had prior video game experience. 
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Table 1. 

Distribution of the Participants by Gender and Game Playing (N=81) 




Female 


Male 



% 

n 

% 

Yes 

40 

49.4 

13 

16 

No 

26 

32.1 

2 

2.5 


Amongst 81 volunteers, 17 were selected for the first interviews. The GPA of these participants ranged 
between 1.73 and 3.9 (M=3.15, SD= 0.66, n=15) and ages were between 19 and 22 (M= 19.65, SD= 0.79, «=17) (See 
Table 2). 


Table 2. 

The Distribution of the Interviewees by Gender and Game Playing 




Female 


Male 


n 

% 

n 

% 

Yes 

9 

52.9 

3 

17.6 

No 

5 

29.4 

0 

0 


Data Collection 

A demographic questionnaire was used to obtain information age, gender, student traits (i.e. year, GPA, 
department etc.) of participants and their game-playing habits. Questionnaire contains two open-ended and one 
dichotomous (Yes or No) questions inquiring about the participant’s house and game-playing habits. 

Semi-structured interview questions were prepared in line with the research questions. After consulting 
with two experts on the questions; a pilot study was conducted with two of the participating students. It was 
concluded that the questions were optimal for the research; hence there were no changes to the questions, data 
obtained from the pilot study was incorporated in the study. The interview parts consisted of three parts: 
introduction, interview questions and conclusion. The interviews with the participants, with the consent of the 
participants, were recorded on a voice-recorder. 


Data Analysis 

Recorded interviews were transcribed which were then chronologically copied on a single text document. 
After studying the transcripts thoroughly, and using the research questions as a guide; codes, categories and sub- 
categories were constructed by comparison of the responses gathered in the transcripts. Utilizing open coding 
(Merriam, 2009); codes, categories and sub-categories were then labeled in congruence with the research questions. 
Codes obtained from the open coding were compared with another set of codes obtained by an expert who has 
conducted open coding on the same transcripts. The final form of the codes was prepared. Later, the codes were 
checked by a colleague. The codes found were arranged on an excel sheet in four columns with the headers: Data 
Source, Analysis, Comments and Frequency. Separate discussion sessions with two contributors were held in order 
to come to accord on the codes, categories and the sub-categories. Disagreements amongst this group were resolved 
by voting until a consensus was reached. The purpose and the scope of the study along with all the codes, categories 
and sub-categories were listed. 
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Procedure 

The online course was gamified before the beginning of the semester. One week prior to the beginning of 
classes, each student enrolled was sent an acceptance latter to the class; the aim of which was to create suspense and 
raise curiosity. Participants were asked to take Bartle Player Type test, which is named after a researcher, Richard 
Bartle, who is considered as the father of the player-types concept. According to their results, they were put into four 
different houses created by utilization of Edmodo: Salamanders, Sphinxes, Centaurs and Leocampuses. 

The duration of the course was 14 weeks. Throughout this procedure students were provided with technical, 
content, and overall practice related support. At the end of the semester, 17 volunteers amongst 81 students were 
selected for semi-structured interviews. Before the interviews, participants were asked for permission for the 
researcher to use a voice recorder throughout the interviews, and asked for consent to use their data for the study. 

Results 

The results obtained from the analyzed interview data show that there are four overarching themes: general 
opinions and the perceptions toward gamified course, learner related themes, instructor related themes and gamified 
course related themes, in this section, each theme with their sub-categories and codes are presented (See Figure 1). 

Figure 1. Visual Representations of the Themes 





General Opinions and 
Perceptions toward 
Gamification 

* Attitude * Motivation * Content-bounded 

* Fun * Immersion * Age-bounded 




Learner-Related 

Themes 

* Learning Style * Background Knowledge 

* Technology Competence * Interest 





Instructor-Related 

Themes 

* Presence * Support 

* Guidance * Instructor Characteristics (Fun & Flexibility) 





Gamified Course 
Related Themes 

* Step-by-step Learning * Originality * Self-Efficacy 

* Meaningful Learning * Mental Break 

* Progression * Feedbacks ^Flexibility * Management 

* Technical Support * Visibility of peers’ comments 

* Content Design (Concise, Clear, Interactive, multimedia use) 

* Relevant Narrative * Competitive Challenge * Win State 

* Competitive Leaderboard * Badge as Feedback * Continuous 

Reward 



6 



General Opinions and Perceptions toward Gamilication 

The study revealed that majority of the students have a positive attitude toward gamification concept. Some 
participants emphasized on the motivative, immersive and fun nature of the gamification process. In order to stress 
on the fun aspect, one of the participants pointed out that: 

“. . .1 think gamification should be used as it makes the class clearer and more fun for 
students...” 

Another participant highlighted to immersive nature by expressing that: 

“..I sent you an e-mail instead, the fact that we are saying that we sent you an owl 
indicates that we have been immersed in this course in my opinion, I mean that it made 
us to be one of the heroes in the story told there...” 

Furthermore, for motivating nature, one participant stated that: 

“.... right at the beginning, 1 thought it was going to be like the CEIT class we had at 
first but when I realized that it was gamification like this I became more motivated. . .” 

Despite such consensus on some issues, the study shows some controversial issues such as whether the 
gamification is an age-bounded and content-bounded concept. The reason of tagging these issues as controversial is 
that the number of the participants claiming the opposite ideas are pretty close. For age-bounded issue, slightly more 
participants agreed that gamification is an age-bounded process. For content-bounded issue, slightly more 
participants stated that gamification is not a content-bounded process. Two of the participants commented on these 
issues: 


“. . .1 think that all courses could be gamified. ...” 

“. . .the age group of the students should be paid some attention. In my opinion it could 

be better if it addressed a younger age group . . . .” 

Learner Related Themes 

The interview analysis demonstrated that the background knowledge and the interest of the participants 
might play a crucial role in the approach of the students toward gamified course. The participants expressed that it 
would be more attractive and fun if the gamified course was based on their interests and background knowledge. 
The focus of most of the participants were on the narrative on which the course was gamified. Some of the students 
were not familiar with the Flarry Potter theme. Flence they expressed that they would prefer the theme and other 
game components integrated for gamification process to be the ones they are familiar with and are interested in. Two 
comments extracted from the interviews illustrate this situation: 

“..Magic thing should change because there may be some students who do not like this 

kind of thing...” 

“..It took a long time for us to get used to the story. . .” 

The study also revealed that majority of the participants had a low perceived technology competency, 
which may have an effect on their perception of online gamified course. Since the course was a technology 
integration course and they needed to use various types of technologies to attend the class online, they were 
expected to be comfortable with technology usage. However, all of the participants who expressed their fear for 
technology usage stated that they got better throughout the course. The comment of one of the participants 
illustrates this assertion well: 

“. . .1 was not in good terms with technology. Thus at first it had scared me a lot. 

However during the course of the course I realized that I think I am successful. . ..” 

The last sub-category synthesized from the interviews is learning style. Majority of the learners expressed 
that they prefer to attend face-to-face learning sessions rather than online education. In their opinion face-to-face 
interaction yields better results in their studies. One of the participants expressed her concern about issue as: 
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. .1 can take notes when the teacher explains it. It becomes more effective when the 
teacher explains it. . 

Instructor Related Themes 

Another reason of the students’ preference of face-to-face discussion, as all of them stated, is the presence 
of the instructor. Since the gamification is a newly adapted method in educational context and since they were not 
comfortable with technology, they expressed their need for the presence of the instructor. Along with the presence, 
majority of them also emphasized on the importance of the instructors’ support and guidance. They stated that they 
needed instructor’s support and guidance especially at the beginning of the semester continuously. However, they 
needed less guidance and support as the time passed. One participant summarized these findings in a sentence: 

“. . .1 was relaxed because I knew that I would get the necessary support and the help I 
needed them...” 

Instructor’s characteristic is another issue most of the participants expressed to influence their perception 
and opinions about the gamification experience. They stated that the instructor who gamified her/his course should 
be a funny and flexible person. Because according to the participants, serious and strictly structured instructor is not 
appropriate for the nature of the gamification. One participants commented on this issue and said: 

“. . .1 like the professor, she is an easygoing professor. Her character goes well with the 
method....” 

Gamified Course Related Themes 

The study proposed that some of the participants were intimidated by the originality of the course design, 
while some of them emphasized on the fact that the course should preserve its originality throughout the semester 
with different types of activities and materials. One participant’s comments on this issue: 

“It could be better, much more enjoyable and attention catching if it had not lost its 
originality through the semester.” 

In order to be original, the participants claimed, the course design should be flexible to adapt to different 
conditions. Also, some of the participants underlined the fact that a flexible environment should be created for 
students to act on their will in such designed course. 

Another structural issue with the designed course is, as most of the participants pointed out, it should be 
designed in step-by-step manner as the participants claimed that they felt incompetent with the course and the 
technology at the beginning. However, as they learn more and get better with the content and the technology, they 
built self-efficacy. According to some participants, presenting the content and the challenges in small chunks helped 
the learners to go through this process more easily in that process. As the two comments selected from the interview 
transcripts accentuates: 

“During the process of the course I noticed that I myself I am successful.” 

“The fact that it progressed step by step was impressive.” 

All of the participants stated that feedbacks from the instructor were crucial through this process. They 
highly emphasized on the importance of the mmediate and direct feedbacks coming from the instructor. Participants 
expressed that the feedbacks they received were helpful for them on earning their self-efficacy without getting 
stressed. Also, immediate technical support they received from the instructor, some of the participants underlined, 
was another important act that helped them go through this process. This whole process should move like 
clockwork; in case it does not, the study showed that the perceptions and the opinions of the students toward the 
gamified course can easily change from positive to negative. That is why management of the course is an important 
factor to allocate time and effort. 

The interview data also demonstrates that the content of the gamified course should be meaningful, 
concise, clear, interactive and with multimedia integration. By meaningful, most of the participants stated that they 
would prefer to learn content that they can use in different parts of their lives as well. Another matter to point with 
the content is that all of the participants liked the idea of mental break within. Irrelevant (generally humorous) 
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videos, pictures or texts in some parts of the content delivered online were used, as mental break; and the 
participants expressed their appreciation for this method. Two comments extracted from the interview illustrate this: 

. .1 have recently realized that the course contents are the things that we use and come 
across in everyday life....” 

“. . .1 would have made the quests more interactive. . .” 

Another highly voiced factor to pay attention in a gamified course is the visibility of the peers’ comments 
and the self-progress. Students appreciate that their progress for each phase/level and the comments of other 
students to the challenges were made available for them in the interface utilized in the gamified online course. 

Finally, the study indicated that relevant narrative, competitive challenges and leaderboards, badges as 
feedbacks to students’ works, continuous rewards and win state are the elements that should be included a gamified 
online course. Most participants express the added enjoyment they provide to the environment. In order to illustrate 
one of these issues raised by the participants, a comment from a participant about the win-state can be shared. 

“The situation of winning made me extremely happy.” 


Conclusions and Suggestions for Improvement 

The results of the study show that the students have positive attitude toward gamification concept as they 
think it is a stimulating, fun and immersive process. This result may be due to the fact that majority of the 
participants play video games as the obtained demographics show, and they may associate this activity with game- 
playing habit. Immersion and the motivation of the participants are the ultimate aim of the gamification process 
(Kapp, 2012; Deterding, et ah, 2011). Therefore, it can be concluded that this application might reach its purpose. 
However, in order to decrease or eliminate the number of the people opposed to gamification for various reasons, a 
longer study with iterations on feedbacks from the participants and design models for gamified educational 
environment might be needed. Considering this need, further study in order to propose a design model for 
educational context with the longer duration and iterations on feedbacks has been conducted as the follow-up study. 

Embedding iterations and designing the gamified experience in different settings might be the solution for 
the contradictory results about whether gamification is content and age bounded concept obtained in this study. The 
results indicate that slightly more participants consider gamification as age-bounded and not content-bounded 
process. This assertion refers that gamification is appropriate for younger age groups but every content can be 
gamified. These results are contradicting to each other as all contents might not be appropriate for younger age 
groups. In order to resolve this contradiction, Kapp (2012) proclaims that the age and the content are not 
determining factors in the success of a gamified experience but how that experience is designed does really matter. 

Moreover, the analysis of the interviews indicates that there are some people-related factors that the 
participants think to be important in gamication process. For participants, there are four main domains that need to 
be considered while gamifiying an online course: learning style, background knowledge, technology competence 
and interest of the participants. Background knowledge and interest of the students should be obtained to be used for 
designing the course; because students get stressed and find it intimidating to work in an unfamiliar and 
uninteresting context. The biggest problem in this study was the narrative selected. Some of the participants who 
were not familiar with and/or not interested in the Harry Potter related theme protested against having to work in an 
unfamiliar or an undesired environment. Henceforth, a detailed learner analysis should be conducted before 
designing the gamified environment. Another reason for the need of learner analysis before designing the gamified 
experience is to acquire more information about learners’ technological competence and learning style. Because, 
surprisingly, a majority of the participants stressed on their low technology competence and fear for technology 
usage for educational purposes. The reason for this astonishment is that on the basis of their birth dates, the 
participants in the study are part of the Generation Z as discussed in the Literature Review section. However, they 
do not have the characteristics of them; in fact, on the contrary, they were quite afraid of the technology. With the 
immediate and continuous support and guidance of the instructor, this problem was aimed to be alleviated, which 
according to the participants worked well. Given this assertion, for the future applications, immediate support and 
guidance especially at the beginning of the process should be given to the students. As a matter of fact, physical 
presence of the instructor in this process, the results suggest, will help learners go through this process more easily. 
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Indeed, although the participants claimed that they would prefer face-to-face education rather than online education, 
and they were not comfortable with technology, as they went through with guidance and support of the instructor 
both face-to-face and online, they become more comfortable with online education and technology. The role of the 
instructor in this process is very important since the participants needed an authority figure to guide through this 
newly integrated process. However, their need for authority figure does not mean that they wanted a strictly 
structured instructor who followed rules. On the contrary, the results indicate that the instructor in such a context 
should be funny and flexible 

Similar to the instructor, the course structure should also be flexible, as results suggest. In a flexible context 
which can adapt to any changing condition such as changing need or demand of the students, students can be given 
the charge of changing the flow of the course structure. For example, the participants preferred to have the flexibility 
of selecting the challenge they wanted to do rather being forced to do them. Another result obtained from the study 
shows that students feel intimidated due to the originality of the course designed. Therefore, guidance and support 
(especially technical support) are very important in the first phases of the course. However, as the participants 
earned more self-efficacy on the content and the technology, this feeling turned into curiosity. After a while, since 
the structure of the course stayed the same, they started to get bored. That may suggest that originality of the 
activities and the methods have a good impact on the perception of the students toward gamified course as long as 
they get the necessary support and guidance at first. Therefore, different kinds of methods and activities should be 
applied on a regular basis in a gamified online course. Another reason of the participants feeling better about the 
gamified course is that content and the challenges were presented in small chunks so they could follow them step- 
by-step. 


In this process, as mentioned earlier, the role of the instructor is quite intense. The participants asked for 
immediate and clear feedbacks as they went through the challenges. Designing such a course, providing necessary 
feedbacks, support (technical, content and activity) and guidance throughout the semester immediately and clearly 
requires a immense management and enormous effort. As the study suggests, if all of these management issues are 
not planned well, the perceptions and the ideas of the students could quickly change from positive to extreme 
negative. 


Another result of the study shows that the participants want to track their progression and their peers’ 
works in the gamified online course. Therefore, in the interface applied in a gamified online course, a progression 
bar showing the students’ progress throughout the course should be placed in a visible place; and in the challenges, 
students can be prompted to comment on a common area where peers can see the comments left by others. It is 
interesting to find that students preferred to learn from each other rather than from the content the instructor 
prepared. Speaking of the content design, the study demonstrates that participants prefer concise, clear, interactive 
and meaningful content with multimedia and mental break integration. For that, content can be prepared in concise, 
clear and interactive manner. For those who wish to learn more, additional resources could be suggested. 
Additionally, if the content is meaningful to the learners, they are more willing to learn. Therefore, even though the 
content may not be relevant to learners’ interests, some samples about how they can transfer the information in the 
other parts of their lives can be suggested. Moreover, multimedia such as video, animation and pictures should be 
integrated in the content with the purpose of either supporting the textual content or for mental breaks. The results 
indicate that as the participants read through the content, they prefer to see some irrelevant content for mental 
breaks. Therefore, some funny videos or pictures or ‘Did You Know...’ information can be presented in different 
parts of the content. However, further study is required to determine where to place these mental breaks in the 
content. 


Lastly, the inteiview results show that participants commented on five of the game elements integrated into 
gamified course. The participants suggested that the narrative be relevant to their interests and background 
knowledge. The challenges and the leaderboards were appreciated by the participants as these created a competitive 
environment. They also stated that they liked to win and earn reward as long as the rewards and the win state are 
continuous. Finally, the participants voiced gratitude on the badges and said they considered them as a new way of 
feedback. These results might suggest that participants like game elements; yet how they are designed and applied is 
an important factor that changes their overall perception and opinion towards the gamified online course. 

In short, as the results point out, the integration of the game elements in the educational context idea is 
attractive for the learners. However, the design of the materials and the course structure is an important issue. More 
studies are needed to elaborate this issue. 
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Abstract 

The purpose of this study is to identify the dimensions of how English as a Foreign Language (EFL) 
students perceive learning with a MOOC, and to identify the effects on instruction when using MOOCs with 
EFL learners. MOOCs have been opening opportunities for learning for global learners. In many cases, 
English is used as the language of instruction, since English is a global language. This means that those 
participating are both native and non-native speakers of English. However, MOOC learners have often been 
analyzed as if they were one group of learners who speak the same mother tongue, in spite of the fact that 
language plays a big role in learning. This study focuses on learners who speak English as a foreign language. 
The study was conducted as a series of studies to develop a MOOC scale and to show the effects of instruction 
with MOOCs. The study found that the Perception of Openness, Self-Efficacy and Willingness to Communicate 
are important dimensions for learning with MOOCs for EFL students, and that the instruction has significant 
effects on the Perception of Openness. 


Introduction 

Education, technology and culture interact with each other in a MOOC learning environment. 
Regarding the nationalities of students using MOOCs, Anzai (2014) reports that when they developed and 
opened a MOOC at Kyushu University, Japan, called Global Social Archaeology from JMOOC in 2014, 799 
people registered, and out of these registrants, 461 were from Japan, while 323 were from countries such as 
Australia, the US, England, China, and the Philippines. Though the language of instruction was English, the 
learners came from a wide variety of nationalities, and they had a rich learning experience with the help of 
subtitles. The completion rate of this MOOC was quite high, recoding 28% (Anzai, 2015). In the MOOC 
environment, the learners’ culture, education, and the MOOC itself represent current educational technology 
interacting dynamically. 

While distance education has been open and online, MOOCs are unprecedented in their huge number 
of enrolled students. Coursera alone documented that there are 16,342,394 learners and 1,488 courses are 
offered (Coursera, November 26, 2015). Capturing this trend. The New York Times (November, 2, 2012) 
featured articles “The Year of the MOOC.” MOOCs offer basically free online courses from elite universities. 
The learners can choose the course they want to register for without taking an entrance examination or fulfilling 
other types of enrollment requirements. There is no tuition, nor fee for the course, though there are cases in 
which learners have to pay for a certificate earned. The MOOC learning style is quite flexible. Learners can 
use various devices such as computers, smartphones, or tablet PCs, so they can control how they learn. From 
this perspective, MOOCs are characterized by their openness (Anzai, 2015b; Bonk, 2015.) 

In some sense, MOOCs provide ESL/EFL learners the opportunity to “study abroad.” One of the ideal 
ways to acquire a language is to live and to study where it is spoken. On the other hand, MOOCs can be a good 
place to learn English as well as specific subject matter even if the learner would be technically classified as an 
EFL learner if he or she were abroad. When students take a MOOC, they may well encounter language stress 
and barriers, since English, a foreign language, is dominantly used in MOOC instruction. With respect to 
MOOCs and ESL/EFL, there have been previous studies. Wu, Fitzgerald and Witten (2014) developed an online 
corpus-based language learning tool utilizing a Coursera MOOC, considering that learners would be motivated 
to improve their knowledge and usage of key academic terms and concepts. Whitmer, Schoilorring, and James 
(2014) studied logfiles, and analyzed students’ responses to two surveys, in a MOOC called, “Crafting an 
Effective Writer: Tools of the Trade,” provided by Coursera in 2013. They reported that 59% of the respondents 
were learning English as a Second Language, and 51% of the students were not enrolled in formal educational 
institutions, and over the half of the learners took this course to advance their professional or career 
opportunities. These studies suggest MOOCs can be used for language teaching and learning. However, there 
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are scarcely any studies which have focused on the perceptions of EFL learners beyond the course evaluation 
and demographic data. 

MOOCs are an evolved form of Open education. Open education is an ideology to open opportunities 
for learners. Learners can participate in a course by a professor from a prestigious university from anywhere, 
and for free of charge. The significant difference from traditional distance education is that MOOC participants 
can communicate with an instructor and/or the other learners in a learning community. Thus, the communication 
in a MOOC is often intercultural with the learners from diverse cultural backgrounds. In this regard, willingness 
to communicate is one of the significant characteristics of learning with MOOCs. 

For these reasons, this study highlights Openness, English Self-efficacy, and Willingness to 
Communicate. 

Openness 

Bonk (2009) defines Openness as “anyone, now, can learn anything from anyone at any time.” 
Elaborating Bonks’ definition, Anzai (2011) conducted an extensive quantitative study on Openness. She 
contends that by integrating Web 2.0 technologies, the traditional class can be extended and expanded to a 
planetary community where learners can participate, interact and collaborate by transcending time and distance. 
She found that there are three factors of Openness: Open Choice, Open Access, and Planetary Community. 
Openness refers to the Perception of Openness by enhancing opportunities for teaching and learning using Web 
technologies. Open Choice refers to options for learning: various choices in learning methods, services and 
support. Planetary Community refers to a learning community where learners can participate interactively 
from a global perspective using emerging technologies. 

English Self-efficacy 

Bandura (1997) defined Self-efficacy as a person’s judgment of his/her capabilities to complete a 
specific task with the skills he/she possesses, so this does not exactly denote the scores from the learners’ 
competence. Zimmerman & Martinez-Pons (1990) claim that Self-efficacy is predictive of learners’ academic 
achievement. In this regard, self-efficacy is important when a learner tries to achieve a goal. This study deals 
with Self-efficacy with respect to English language learning. For example, “I can understand the lecture in 
English.” 

Willingness to Communicate 

“Communication” is a significant characteristic with MOOCs. Comparing learning with MOOCs and 
traditional distance learning, Bonk (2009) articulates that in a MOOC, “Learners are not alone anymore.” 

They can ask and answer questions, and communicate with each other in a learning community by connecting 
with other learners (Kop et al., 2011). Anderson (2003) identified that leamer-leamer interaction and learner- 
teacher interaction are important kinds of interaction, in addition to learner-content interaction. In the context of 
language learning, McCroskey and Baer (1983) developed the construct of Willingness to Communicate as the 
first language acquisition. Soon after this, researchers in ESL recognized this is an important construct for 
learning a language. There are two approaches for Willingness to Communicate; trait-like and situational. This 
study analyzes Willingness to Communicate from a situational perspective, so that perception of Willingnes to 
Communicate may change according to their experience. The construct, for example, includes “I would like to 
be friends with foreign people.” 


Methodology 

The study was conducted in Tokyo with 60 Japanese university students on September 27 and 28 in 
2014. They were two groups: the MOOC group and the Conventional group. These two groups were the same or 
similar in tenns of learner characteristics such as age, gender, EFL proficiency, preference for digital devices, 
and computer experience. There were no statistical differences between these two groups in terms of these 
learner characteristics. 

Regarding the instruction, two types of 60-minute classes were designed as an “Introduction to the 
Class,” having the same purpose: to understand Open education. The Conventional group was told that they 
would use a textbook, whereas the MOOC group was told that they would use MOOCs as the primary learning 
material. Figure 1 is a photo from the MOOC class, whereas Figure 2 is one from the Conventional group. As is 
shown in Figure 1, the leaners were given time to explore the MOOCs using their smartphones during the class. 
On the other hand, as is shown in Figure 2, learners were listening carefully to understand Open education, 
which is a traditional lecture type. The survey which consisted of 84 items was given to the students after the 
instruction. 
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Figure 1. Exploring MOOCs in the MOOC group 



Figure 2, Lecture in the Conventional group 

Results and Discussion 


Openness 

A series of ANOVA were conducted on the 37 survey items of Openness. As a result, there were 10 
items which showed significant differences between the two groups. 

Table 1 . Items which had significant differences in Openness 

1 . Flexibility 

(Regarding EFL) We can remove time constraints, (p < j05) 

(Regarding EFL) We can remove spatial constraints, (p .05) 

(Regarding EFL) We do not have to pay for learning, (p < .00 1 ) 

(Regarding EFL) We can participate in a global community, (p < U5) 

(Regarding EFL) We can overcome physical distance, (p < .05) 

2, Personalized learning | 

(Regarding EFL) It is accepted that our learning ability varies depending on the learner, (p < j05) 

(Regarding EFL) We can learn depending on our needs, (p < .05) 

(Regarding EFL) We have various curricula, (p < .05) 

(Regarding EFL) We have a wide variety of content in regard to study support services, (p < .05) 
(Regarding EFL) We have a wide variety of methods in regard to learning support services, (p < m 


As is shown, the learners in the MOOC group perceived that by integrating MOOCs, learning had 
become more flexible by reducing the barriers from time and space. They also perceived that they could 
participate in the Planetary Community where they could interact with other learners, which contrasts with the 
views of those in the Conventional group. Thus, they can remove time constrains; they can remove spatial 
constrains; they do not have to pay for learning; and they can overcome physical distance. Furthermore, the 
MOOC group perceived that learning could be more individualized by having more choices for learning, which 
again contrasts with the perception of those in the Conventional group. In other words, the MOOC group felt 
that they could learn depending on their needs, have various curricula, have a wide variety of content in study 
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support services, and have a wide variety of methods in learning support services. In sum, the students with a 
MOOC perceived a substantial boost in their perception of Openness compared with the students in the 
traditional group. 

Self-efficacy 

There were nine items to measure Self-efficacy. Out of these items, two items showed significant 
differences: “I can do well with the questions and assignments for the class” with F( 1 , 58) = 5.00, p < .05, and “I 
can manage the level of the class” with F( 1 , 58)=4.09, p < .05. With both items, the MOOC group had 
significantly lower scores than the Conventional class. This would mean that the students were not sure whether 
they could handle the MOOC class or not, since MOOCs provide authentic learning opportunities and they were 
not accustomed to online learning. 

Table 2. Items that showed significant differences in Self-efficacy 

I can do well with the questions and assignments for the class, (p < .05) 

I can manage the level of the class, (p. < _^5) 


English Self-efficacy 

There were nine items to measure English Self-efficacy. None of them showed significant differences 
between the MOOC group and the Conventional group. These items were, for example, “I am good at English,” 
“I can do well in English listening comprehension,” and “I can do well in English writing.” It would be 
reasonable to assume that a 60-minute Introduction to the Class would not make much difference, and the effect 
of the two kinds of instruction need additional time for the implementation of the whole syllabus. In addition, 
when the students’ comments were analyzed, it became apparent that the students want English support to 
understand the lectures. For example, a student wrote that “I want to have Japanese translation with a MOOC.” 
Currently there are MOOCs with subtitles, so EFL researchers need to study what is an effective way to 
integrate subtitles to enhance the learners’ understanding. 

Willingness to Communicate 

There were 15 items which measured Willingness to Communicate. Two items showed significant 
differences: “I would be uncomfortable if we had international students in our class” withF(l, 58) = 7.08,/? 

< .05, and “I do not want to talk with international students” with F( 1 , 58) = 3.112, p <. 10. As for the former 
item, the MOOC group showed significantly lower scores (M= 2.03, 579=1.09) than the Conventional group 
(M=2.79, 579=1.10). The latter item also showed the same tendency. The MOOC group showed lower scores 
(M=2.19, 579=1.18) than the Conventional group (M=2.79, SD=1.45). Thus, the results indicate that the MOOC 
group had higher Willingness to Communicate than the Conventional group. The items which did not show 
differences include “I want to participate in activities in English to help foreigners,” “I want to be friends with 
foreigners,” and “I want to communicate with strangers through SNS.” Therefore, it seems that significant 
differences appeared in a rather passive tone of communication rather than positive tones such as “want to.” 

The MOOC group showed positive orientation toward Willingness to Communicate. To analyze the students’ 
comments, some of the students showed anxiety about having to write and speak in English. One student, for 
example, noted that “I would like to have 24 hours online support using Skype or Line, when questions arise.” 
Promoting the idea that students become “an intercultural speaker,” as opposed to simply “a native-like speaker 
of English,” may encourage students to communicate more freely and with less anxiety. However, it was only 
the Introduction to the Class; the implementation of the whole syllabus may bring significant differences in 
more items. 

Table 2. Willingness to Communicate 

I would be uncomfortable if we had international students in our class. (p<.0.5) 

I do not want to talk with international students, if I do not have to. (p <. 10) 


Conclusion 

This study highlighted Openness, Self-efficacy and Willingness to Communicate as important 
dimensions for EFL students that participate in MOOCs. As a result, it was found that in the Introduction to the 
Class, instruction with MOOCs has a positive influence on perception of Openness in general. The students 
perceived that by integrating MOOCs, the method of study and learning could become more flexible and 
personalized, fulfilling their individual needs. For EFL students, this recognition — that there are various ways to 
access quality education free of charge — is quite important. According to Anzai (20 11), perception of 
Openness will result in enhancing students’ EFL proficiency, particularly English listening proficiency. Thus, we 
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can expect MOOCs to be useful in English language learning. Only the introduction of MOOCs may make a 
difference in the perception of Openness. Full implementation of the whole syllabus is needed to find out how 
MOOCs influence the perception of Willingness to Communicate after participating in a MOOC community, 
and how it influences English Self-efficacy, which is predictive of English language enhancement. 

Many studies suggest that “MOOC learners are typically well-educated, more affluent, mostly from the 
developed world, and male” (Guzdial, November 25): 91% male, 73.3% from OECD countries, and over 50% 
had graduate degrees (Balch, 2013). The studies usually consider gender, educational background, and economic 
and social status of the countries. These are important learner characteristics to consider, but we must not forget 
the language aspect. First of all, if the participants do not understand the MOOC, it will not mean much to them. 
For MOOCs to be truly open for global learners, we need more studies from various language perspectives. 
MOOCs should be beneficial for all learners around the globe. 
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Abstract 

Online games have become one of children’s first choices for both leisure and learning activities. This proposal 
presents case studies of two children who were observed and interviewed while playing a popular virtual world 
environment. Interviews, observations and video recordings of game playing were collected and analyzed. The 
results from this experience may inform teachers, parents and instructional designers about children’s motivation 
and engagement levels while playing educational online games. 

Introduction 

More than 150 million Americans play video games (Entertainment Software Association, 2015). With 
the advance of technology, digital games are becoming popular technologies among young people. In a Kaiser 
Family Foundation study, more than two thousand children and adolescents, aged 8 to 18 years old, were 
surveyed to understand the role of media in their lives (Rideout, Foehr, & Roberts, 2010). The findings showed 
that 60% of children and adolescents in the US play video games on an average day. In addition, the use of 
computers for play among children and adolescents is elevated, with 91% of boys and 93% of girls now playing 
games online (M2 Research, 2010). Thus, it is important to investigate how children are using (Wood & 
Willoughby, 2008) and interpreting these technologies. 

Given the popularity of virtual worlds with children, recent research has been conducted to examine 
potential learning outcomes supported by these technologies. In general, positive experiences have been reported 
regarding the use of virtual worlds and gaming technologies for learning (Barab, Arid, & Jackson, 2005; Barab, 
Sadler, Heiselt, Hickey, & Zuiker, 2007; Barab, Zuiker, Warren, & Hickey, 2007; Kafai, 2010). These studies 
have indicated that children are usually engaged and motivated to use these technologies. In terms of content 
implicit to a game, a few studies (Ke 2008; Pareto, Haake, Lindstrom, Sjoden, & Gulz, 2012) have indicated 
improvement in children’s attitude toward a subject area, such as mathematics, as result of game playing. 

Even though games present characteristics and attributes that can contribute to instruction and learning, these 
tools need to be examined with caution. As a matter of fact, both player’s intrinsic motivation to playing games 
and game characteristics may have significant impact the extent to which these individuals engage with 
educational games. Whereas it is certainly intriguing to examine how players devote different levels of effort and 
persistence while playing educational games, it is important to investigate their behavior while engaged in such 
experiences through the lens of motivation theories (Deci & Ryan, 1985; Ryan & Deci, 2000). As a result, such 
study may shed more light onto how learner’s intrinsic motivation and attainment value to play games influence 
persistent, effort, and learning. 


Theoretical Perspectives 

Motivation has been considered a key element for academic and intellectual success and this explains the 
great deal of research on motivation in the last decades (Dornyei, 1998). The high interest in motivation led to the 
creation of a variety of conceptual frameworks that define this construct from different perspectives. But 
regardless of the theoretical foundation, scholars usually have common objectives, which are to refine the 
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theoretical concept and/or to create guidelines about how to address students’ different levels of motivation within 
a single educational setting. 

One of the common, naive mistakes in attempting to define motivation - and it still remains unnoticed by 
many scholars - is that motivation is oftentimes described and addressed as a stable construct. Being influenced by 
a variety of internal and external factors, personal and situational interest (Urdan & Turner, 2005), motivation may 
not be consistent over time and across contexts. Indeed, motivation is a complex concept, dynamically affected by 
both intrinsic and extrinsic contingencies, all of them responsible for driving one’s engagement, persistence and 
efforts devoted to a task (Ryan & Deci, 2000). Motivation has also been divided into subcategories: intrinsic 
motivation and extrinsic motivation, which represent two ends of a continuum. Intrinsic motivation is defined as 
the propensity of developing a task due to interest, enjoyment and satisfaction. Conversely, extrinsic motivation is 
fostered by external factors as rewards, punishments, threats, ego-involved learning or any type of external value. 

Methodology 

The purpose of this study was to examine young children’s experiences in an online gaming environment 
and to observe their reified motivational levels and attainment values through persistence and effort applied to 
educational games. Given that the games covered counting and addition skills as well as geometric concepts , 
instances of learning in Mathematics were also studied while children interacted with a popular commercial virtual 
world marketed for all ages. A qualitative case study (Stake, 1995) was conducted with a small group of young 
children participating in an after-school program. A total number of six children participated in the after-school 
program. Participants aged between 7 to 10 years old respectively at the time of data collection. Five of the 
participants were girls and only one was a boy. All participants were Caucasians from middle-income families. For 
the purpose of this paper, only two of the participants, i.e. Elizabeth and Rachel, were selected to delineate the 
cases studied. During game session in the after-school program, participants were asked to play specific math 
games for short period of time, but they also had autonomy to select games or activities of their choice once they 
finished playing the math games. 


Data Collection and Procedure 

To gather the data, 10 one-hour long sessions of game playing were held in an after-school program at a 
private school where both children were enrolled. The data collected from these visits were of a qualitative nature 
(e.g., informal and semi- structure interviews, observational field notes and video-recordings). Interviews were 
conducted to gain knowledge on how children understood and interpreted the research phenomenon. Close and 
open-ended questions were used to design an interview protocol to clarify information and generate descriptions 
of participants’ perceptions and experiences (Roulston, 2010). Besides traditional methods of data collection, 
digital technologies were used in this study to provide new sources of data for qualitative researchers (Saldana, 
2011). Taking advantage of digital technologies to better understand the research phenomenon, we implemented 
audiovisual recording methods to capture participants’ interactions within the virtual gaming environment. Audio 
and video screen recordings were also collected during the after-school program and were considered the primary 
data source to inform the findings of this study. 


Data Analysis 

Visual analysis methods were implemented to analyze the data gathered from this study. A particular 
qualitative data analysis strategy was used to construct the cases: interaction analysis (Jordan & Henderson, 1995). 
This analysis technique was conducted to examine children’s interaction with the virtual world and games. Other 
practices borrowed from Grounded Theory methods (Charmaz, 2006; Corbin & Strauss, 2008) were also 
implemented to identify recurring patterns and themes throughout data collected. After an initial analysis of the 
data set, common games children played were identified as well as their way of playing in this virtual world. Video 
transcriptions from the video recordings included annotations of screen-based activities, such as objects that 
participants were manipulating within the game or mouse cursor movements. 

Following Jordan and Henderson’s (1995) guidelines, video data was chucked based on the events 
happening within a game session. Gerunds (e.g., “playing Bits and Bolts” or “visiting penguin’s igloo”) were used 
to label these events to provide a sense of action and sequence to the data (Charmaz, 2006). Only relevant events 
were transcribed. Jordan and Henderson (1995) also suggested the analyst attends to “segmentation”, especially 
the transition from one segment of an event to another. These procedures were followed to observe any 
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segmentation patterns when participants were about to disengage from playing a game. These segmentation 
patterns were often indicated by repeated mistakes when a participant was logging out or when moving the mouse 
cursor near the exit button. 

Findings 

A typical game session involved children being asked to log into their virtual world account and be asked 
to play specific math games for as long as they could sustain their engagement - average of seven to twelve minutes 
depending other math game - before seeking other games or activities of their choice. All participants appeared to 
enjoy playing in the virtual world, but they were not always motivated to play math-related games. Most children 
showed signs of disengagement with one of the math games, i.e., Bits and Bolts, because the math task involved in 
the game was explicit. Participants had to select and count the correct combination of bolts to meet the targeted 
number displayed on the screen. Children showed their disengagement with this particular game by negotiating the 
amount of time they would play the game (e.g., “four minutes”) or by voicing displeasure to play it (e.g., “not Bits 
and Bolts. I don’t wanna play Bits and Bolts”). Still, some participants continued to play Bits and Bolts for the 
points acquired in the gameplay. The points were converted to virtual money (coins) that participants could spend 
to buy accessories for their avatar or virtual home or even their virtual pet. 

Another math game played by participants was Pufflescape. In this game, participants practiced 
mathematical knowledge (e.g., geometric concepts such as angles and shapes) by helping their puffle, i.e., virtual 
pet, to escape from an icy cave. Players often used geometrically shaped blocks of snow to build ramps and 
collect a key to open the gate that leads to next phase. Besides the key, the player could also get “O’ berries” to 
score more points in the game and earn coins that they could spend in the virtual world. The berries required more 
effort and strategic planning in the game. Rachel, one of the participants, did not devote much effort to the game 
because getting the gate key was enough to pass to the next level pass to the next level. On the contrary, another 
participant, Elizabeth devoted much more effort and persistence to get all the rewards (both berries and key) in 
every level before finally going to subsequent levels. An excerpt from this case is presented below. 

Excerpt from the Case of Elizabeth and Rachel 

We paired Elizabeth with Rachel to walk us through the game as means to provide insights of their 
understanding of the game. They played the game with the same puffle and just took turns between levels. While 
playing the first level of the game, Rachel provided instructions on how to move the puffle around: “you have to 
use the arrow keys. . . collect. . . roll down. . . roll off. . . get the key and roll pass the ice.” We asked Rachel to wait 
before she rolled the puffle through the gate door, but she did not attend to our request. 

Elizabeth started playing the second level determined to collect all berries. While Elizabeth kept playing, 
Rachel provided her with directions, which sometimes clashed with Elizabeth’s actions during play. For instance, 
Elizabeth was trying to collect the three berries, which were fairly difficult to reach and collect in the game: 

Rachel: That way. No, you can’t (Elizabeth used the mouse to move the lever). You’ve got to use the 
arrow keys. 

Elizabeth: Woohoo (Elizabeth set up the ramp while the puffle was on the opposite side of the ramp) 

Researcher: But now, you cannot go there. 

Elizabeth: Oh, yeah! 

Elizabeth put the lever on its initial position and rolled the puffle over to collect the key. Elizabeth wanted 
to collect all the berries for this level, but Rachel disagreed: 

Elizabeth: Now, I have to get all the coins. 

Rachel: No. No, you don’t. (. . .) You don’t need to do that, seriously. 

Elizabeth: Oh, I seriously need to. (. . .) 

Researcher: She got it! 

Elizabeth: I just schooled you. 

Rachel: Those are stamps anyway (Elizabeth called the berries “coins” and Rachel referred to them as 
“stamps”). 

As a result of collecting all the berries, Elizabeth unlocked the extreme levels and thought she had a 
second turn in the game. Upset with the idea of Elizabeth having a second turn, Rachel shouted: “no, no, no fair” 
and quickly took over the computer to have her turn. Seeing the third level of the game, Elizabeth begged Rachel 
to let her play during Rachel’s turn: “Oh, I love that one! Please? Please let me do it and then she’ll get to have a 
second turn.” Rachel ignored Elizabeth’s request and continued playing the game. 

Elizabeth and Rachel had different game styles when playing Pufflescape, which led to some tension 
between the two. For instance, Elizabeth commented that Rachel “did it wrong” because she did not collect all the 
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berries in a level. Meanwhile, Rachel seemed aggravated with Elizabeth’s need to collect all the objects in the 
level as shown in the following conversation: 

Elizabeth: I gotta do that ‘cause I gotta get that coin, don’t I? 

Rachel: Nope! 

Elizabeth: Oh, I want to! 

Given that Elizabeth sought to collect all the berries in the game, she was also more knowledgeable than 
Rachel regarding game features. For instance, Elizabeth told Rachel to “press the cheats”, as she called the Heads- 
Up Display of this game that allows them to see the math involved in the game and help them to find their way 
out of the fifth level in the game. When Elizabeth was asked about “the cheats”, she said: 

Elizabeth: It means to get that right there 'cause that was where the key was [meaning that the goal was to 
get the key] and so you put that [referring to the block of snow] right there [referring to the red scribble where the 
25 degree angle was located], put that right there (...) and then you put like a ramp and go like choo and you go 
down like that and you go there. 

Elizabeth’s drive to collect all the berries provided her with the opportunity to understand the purpose of 
the cheats. Consequently, it exposed her to the implicit math content presented in the game. The cheats served to 
help Elizabeth achieve the game goal during play as well as the external goal of earning coins to be spent in the 
virtual environment. 

Overall, Rachel indicated that berries were not needed to succeed in the game and advancing the levels of 
Pufflescape was enough. As for this specific game, she did not have as many opportunities to practice her 
geometrical concepts as Elizabeth, who tried to figure out a way to get (almost) all the coins and the key. The 
latter had an achiever approach to the game, i.e., a high motivation to achieve rewards just for the prestige of 
having them. 


Discussion and Implications for education 

Both math games examined in this study were designed to promote both player’s engagement and 
learning. To a certain degree, these games covered math content as part of the game play. Playing Bits and Bolts 
involved practice of basic arithmetic content, such as counting and addition. Playing Pufflescape involved 
practice of geometric content, such as angles and shapes. The exposure of content in these games was also 
different. In Bits and Bolts, the content was explicit. Most children identified it as a math game and noticed 
addition as part of the game play. In Pufflescape, the content was implicit. Children were unsure of what and if 
academic content was involved in the game. These factors contributed to children’s engagement and sustained 
attention to a game over another. The academic content in Pufflescape was hidden. Most geometric content in the 
game was presented through a Heads-Up Display (HUD). By clicking on HUD, children were exposed to angles, 
theorems and formulae to solve puzzles in the game. Children applied this content as they manipulated levers and 
icicles to make a ramp. Pufflescape provided more optimal challenge to foster a flow experience 
(Csikszentmihalyi, 1990) than Bits and Bolts. When facing challenges to collect all the berries, children depended 
on HUD as a reference tool. Children also had more flexibility and autonomy (Rigby & Ryan, 2011) playing 
Pufflescape. For instance, children had the choice to collect or not all the berries to level up. 

Teachers and parents could use games, such as Pufflescape, to start a conversation around topics and 
content children are learning while playing the game. Moreover, games can be used to assess performance on 
task. If children were taught specific strategies in the classroom, teachers could use the game to assess children’s 
implementation of those strategies. Finally, findings from this research might inform the use of these technologies 
in informal and formal settings. The results from this experience may enlighten teachers, parents and instructional 
designers about children’s actions and behaviors in an online gaming environment. Understanding children’s 
experiences with these technologies might help instructional designers and teachers to better plan for the use of 
games for learning. 
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Abstract 

This paper discusses findings of a pilot study on the applicability of design thinking process in higher 
education in Afrika. The results show that local knowledge and approaches are necessary for the successful 
application of design thinking, in Afrikan higher education. We argue that this is of particular interest to researchers 
and policymakers involved in higher education reform in Afrika, and around the world. 

Introduction 

Adoption of educational theories or strategies is no longer limited by geographical regions, especially given 
the existence of varied technologies that provides more opportunities for knowledge sharing, and collaboration. 
Researchers around the world share practices that work in different educational settings and adopt methods and 
practices that are effective in other countries. As Afrikan governments in particular, increasingly encourage adoption 
of models and techniques of education that are successful in North America and Europe (Woldegiorgis, Jonck, & 
Goujon, 2014) in order to reform higher education on the continent, it becomes relevant and critical to examine 
various method, models and theories and question their applicability and relevance to the the Afrikan continent. One 
of such method is Design Thinking (DT), a human-centered approach to design that was coined and developed by 
the founders of IDEO, a California based design firm. IDEO’s innovation is "a blend of methodologies, work 
practices, culture, and infrastructure" that can work everywhere (Kelly & Littman, 2001; Gobble, 2014). It’s "a way 
of finding human needs and creating new solutions using the tools and mindsets of design practitioners [. . ..jrelying 
on the natural- and coachable-human ability to be intuitive, to recognize patterns, and to construct ideas that are 
emotionally meaningful as well as functional" (Kelley, 2013). 

Proponents of DT contend that the process can be taught, and can work everywhere (Goggle, 2014). 
Particularly because DT empowers individual to think creatively when faced with challenges, to collaborate, and 
welcome others’ perspectives (Keller, 2013). Presented as an approach to enable and innovation (Kelly, 2001), few 
in the United States have challenged the use of DTin education. However as design thinking gains international 
prominence, perhaps as a result of a growing number of students from different cultural background educated in the 
United States who return to their country of origin, a close examination of this process may be necessary to 
empower and meet the needs of the target audience. 

Research Questions 

The concept of DT has been discussed by researchers and practitioners from fields such as information 
technology, business management and education in the United States. Yet, little is known about the applicability of 
this model of education in Africa. To fill this gap in DT research, this paper addresses the applicability of this 
process in an African educational context. The current study is guided by the following two questions: Can design 
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thinking be applied in Sub-Saharan Afrika higher education systems? How can design thinking process be used in 
higher education in Sub-Saharan Africa? If design is understood as an approach to solving issues, or facing 
challenges, is there a similarity between the sub-Saharan design approach and design thinking as practiced in the 
US. 

Methods 

A case study approach was chosen because it allows the researchers to gain in-depth comprehension of the 
meanings and situations of the target population (Hancock & Algozzine, 2006). A case study methods is appropriate 
when examining questions related to experience in a specific environment (Patton, 2002b; Creswell, 2013). In 
addition, a case study approach offers the opportunity to conduct micro and meta level inquiries, leading to more 
holistic analyses (Qiu & Yang, 2010). While the purpose of this pilot study is not to develop generalizations, we 
hope that the results of this case shed light on the adoption and implementation of practices, and techniques in 
different contexts and cultural settings. 


Participants 

This pilot study focuses on African students from Nigeria and Cameroon who attend a high ranked 
university in the United States. All respondents previously studied law and graduated from universities in their 
country of origin. The selection of participants from Nigeria and Cameroon was in large part because of the 
limited number of Africans at the chosen university. Moreover, the two countries share similar cultures but have 
different colonial influences on their educational systems, whereby Nigeria was colonized by Great Britain and 
Cameroon colonized by France and Great Britain. Three participants were selected for interviews, using snowball 
sampling method. In terms of gender, two participants were male and one a female. Politically, all participants 
were citizens of their countries of origin; two worked as lawyers in their country of origin, and one participant was 
engaged in higher education reform and political activity in his country. 

Data collection 

Our data were collected through semi-structured face-to face interviews which allowed us to gain access 
into interviewees’ “inner perspective” (Patton, 2002, p. 341), and provided us with the opportunity to relate with 
them and gain a deeper understanding of their words in terms of emotions, views and intents. The interviews 
included questions such as: In your classes, have you ever been given problems/cases to solve? Can you describe 
how you approached or tried to solve the problem or case? Can design thinking process be taught in your country? 
Would teacher/students embrace this process? Why/why not? This type of interviews also allowed us to cover 
specific topics (Rabionet, 2011). 

Three interviews were conducted and audio-recorded. The interviews were approximately 40 minutes in 
length.. All interviews were conducted in English. It is however worth acknowledging that language being a 
representation of reality, and an instrument for communicating one’s truth (Qiu & Yang, 2010; Spradley, 1979), the 
use of participants’ primary language could have deepened the understanding of participants’ response. Indeed, 
participants had to translate from their primary language to English especially when it came to proverbs they used 
during the interviews to convey an idea or concept to the researchers. 

Data Analysis 

Content analysis method was used to analyze data in this study. Audio recorded interviews were 
transcribed, enabling all authors to access the data. Using the coding scheme described by Graneheim and Lundman 
(2004), we carefully analyzed the interviews, generated meaningful units, coded, organized codes into categories 
and converted categories to themes. We used an iterative approach in the coding, categorization and identification of 
themes. The analysis focus on the applicability of DT processes in Afrika higher education. Contribution of all 
authors in the coding of all transcripts provided inter-rater reliability. 

Results 

Three themes from the review of data are discussed in this paper: different educational system, ideas for 
applying design thinking in an Afrikan context and design thinking embedded in Afrikan culture. 
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Different educational system 

Design Thinking was seen as dependent on the educational systems of the two countries. Participants in this 
study argued that the educational system of their respective countries was too different to accommodate design 
thinking application in higher education settings. Participants expressed such difference by referring to teachers’ 
teaching approaches, the influence of colonization on Afrika educational system, and the adoption by Afrikan 
countries of the colonizer’ approach to knowledge transfer. Two examples of meaning units that support this finding 
are provided below. 

“The teacher comes to class, just dictates the courses and you just take notes; when he dictates courses he 
can give a subject, tell you that last year I gave this, and I expect students to answer this way. He does not go 
deeply.” (Cameroonian 1) 

“When the colonialists came, mostly they saw us as inferior, they saw our culture as inferior, that's why in 
eastern African countries you have the assimilation. They were not even trying to indirectly rule Africa, they were 
trying to make French citizens out of Africa because they thought we were just inferior. Right? So they always 
talked down on us, and then the system they have in England, which is the mode of passing on instructions was at 
some point similar. And we are stuck with the notion that Africans are supposed to be inferior”. (Nigerian 2). 

Ideas for applying design thinking in Afrikan context 

Despite the difference in educational systems, participants in this study believed that there could be ways to 
apply design thinking in an Afrikan education context. Suggestions made by respondents for applying design 
thinking in Afrikan contexts included: adopting Afrikan approach to the introduction of new ideas or concept, 
working with a new generation of teachers, and starting to apply the concept at the elementary level to facilitate 
cultural transition. Examples of meaning units that support this theme are provided below. 

“What you can do it is to use the African approach. Find an elderly person, I mean a professor who has 
experience and is well respected among his peers. Meet with him, and convince him about the usefulness of this 
concept. Once he is convinced, he can go with you now, and be the one presenting the concept to his colleagues” 
(Nigerian 1). 

“Starting with primary school it is even better because teachers will not see them, kids as a threat. Kids like 
10 years old; start with primary five. For kids it is what you teach them that they do; it will be easier. So at the 
university level, you will let teachers get use to that, and you will let teachers see that coming. This can help in the 
cultural transition” (Nigerian 1). 

“Maybe the new generation of teachers who study in the US can change things. Because now we see a lot 
of students who came to the US to study, come back, and they are trying to change things”. (Cameroonian 1) 

Design thinking embedded in Afrikan culture 

The analysis of data also reveals that design thinking process is already embedded in Afrikan culture. 
Participants argued that “we did have something like that in our culture. It’s been there, it’s not just been applied at 
the university level” (Nigerian 1). A similarity between design thinking process and Afrikan approach to solving 
issues was also acknowledged by participants as one stated that “There is a little bit of similarity. The fact that we 
give everyone the opportunity to speak. The similarity is that you listen to others, but the elders still have the final 
say” (Cameroon 1). 


Discussion 

This pilot study explored the applicability of design thinking process in two Afrikan countries. The data 
discussed in this paper show that design thinking process cannot be applied in Afrikan educational system without 
adjustment and contextualization. The experience in Afrikan countries which is greatly impacted by colonization is 
different from the USA experience in terms of educational systems. The results also reveal that the application of 
DT in Afrikan education can be possible. However, Afrikan approaches to introducing new ideas has to be 
integrated in the application/adoption process. Furthermore, the study revealed that DT as a process is not a new 
concept for Afrikans. DT process is inscribed in Afrikan culture, and recognizing that fact might help in its 
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application. These findings align with previous studies investigating the application of teaching strategies that work 
in the West in other cultural context (Nguyen, 2008; Thanh, 2014). Thanh (2014) argues that the importation of 
teaching strategies from the West should be done with careful consideration of the socio-cultural factors specific to 
the targeted context since learning is not a standalone event. A similar consideration should be adopted by Afrikan 
education reformers. For DT or any other strategy to work in Afrikan education context it is necessary to examine 
all factors that could support or hinder the application process. In other words, design thinking as an innovative 
approach to education will not be successful in Afrika, unless local knowledge, strategy or approach to solving 
problems is engaged and considered in the application of this process (Bayeck, 2015). Engaging local knowledge 
and approach to solving problems and introducing new concept is important for the empowerment of all and cannot 
happen without recognizing and acknowledging the significance of locals’ insight. The race into the future, and the 
acceleration of learning cannot occur without creating opportunities for all to contribute to their learning and 
knowledge construction. 
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Abstract 

The MOOC phenomenon, which was an idea a short time ago, turned into practice and drew much 
attention as a disruptive innovation. On this ground, the purpose of this study is to explore and explain 
MOOCers’ attitudes and preferences as well as the problems they face during a MOOC. In order to gather data, 
an online questionnaire was conducted in a Facebook group of a five week long hybrid MOOC, initiated in 
January 2013. The questionnaire was provided to Facebook group members (nearly 5000) and only of 161 out 
of all members voluntarily completed the questionnaire. This study revealed that MOOCers perceive these mega 
classes as a lifelong learning opportunity. Additionally, most of the MOOCers were satisfied with their MOOC 
experience and have had positive impressions about MOOCs. A great majority of MOOCers plan to join another 
MOOC in future and it proves that MOOC hype will go on for some time. 

Key words: MOOCs, Hybrid MOOCs, MOOCers, MOOCers’ preferences, heutagogy 

1. Introduction 

The impact of the Web, the Internet and online access changed the way people interact with one 
another and information resources (Anderson, 2008). Developments in computer networks increased and 
diversified learning experiences while lessening individuals’ dependency on institutions (Kop, 201 1) and 
enabled individuals to seek for information and knowledge as a part of their lifelong learning journey. As a 
result of developments in educational technology, education was forced into transformation (de Waard, et al. 

201 la) in which “openness” appeared as a key concept. 

The concept of openness is a term that has been in constant negotiation (Cormier and Siemens, 2010) 
and has changed in the last ten years (McAndrew, 2010). It has become a complex code word for a variety of 
digital trends and movements (Peters, 2010). For open education movement, “openness” means ensuring that 
there is little or no barrier to access for anyone who can, or wants to, contribute to a particular development or 
use its output (SURF, 2013). As a result of open education movement, Open Education Resources (OER) and 
following that MOOCs appeared in the context of open education as a way to open the gate for different 
learning opportunities. 

MOOCs, as a disruptive innovation, have been perceived as a learning opportunity by learners and as a 
threat by brick and mortar higher education institutions. In An Avalanche is Coming, it was stated that “a new 
phase of competitive intensity is emerging as the concept of the traditional university itself comes under 
pressure and the various functions it serves are unbundled and increasingly supplied, perhaps better, by 
providers that are not universities at all” (Barber, Donnelly, & Rizvi, 2013: p.l), but by private providers and 
MOOC platforms. All in all, recent developments in educational technology and the demand for continuous 
learning inspired MOOCs as a rising learning trend in 21 st century education. 

2. Overview of MOOCs 


2.1 Defining MOOCs 

The development of MOOCs is rooted within the philosophy of open education movement, in which it 
is believed that knowledge should be shared freely, and the desire to leam should be met without demographic, 
economic, and geographical constraints (Yuan and Powell, 2013). On these bases, a MOOC is an online course 
with the option of free and open registration, a publicly shared curriculum, and open-ended outcomes. Most 
significantly, MOOCs build on the engagement of learners who self-organize their participation according to 
learning goals, prior knowledge and skills, and common interests (McAuley et al, 2010). MOOCs ascribe to the 
principles of universal access: it is available to anyone with Internet access. Enrollment sizes usually tend to be 
high; pedagogically, first generation MOOCs in particular embrace an open, social structure, and a 
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constructivist, connectivist manner of knowledge production. In short, learners join, participate, and also 
withdraw at high frequency (Koutropoulos et al, 2012) as a natural result of open and flexible learning system. 
MOOC is an acronym for “Massive Open Online Course”. In this definition, “M” refers to the word massive. 
Even though it is primarily used for the number of participants, Levy (201 1) states that massive also covers 
participants’ diversity, the kinds of backgrounds and experiences, the communication tools, the web 
technologies, the amount of distributed knowledge and the complexity of the distribution, the overwhelming 
width and depth of discourse among the participants, the multi-modal nature of the discourse, and finally the 
massive amount of time needed to manage and organize. The second letter “O” refers to the word open. 
Openness means to be free to join, create, interact, analyze, and reflect according to participants’ own learning 
needs (Koutropoulos et al, 2012). Openness embraces all levels of engagement, with no barriers between in and 
out (Downes, 2008). Openness and flexibility help to maintain the free flow of information through the 
networks, and encourages a culture of sharing and a focus on knowledge creation (Mackness, Mak & Williams, 
2010). 

The other two letters, “O” and “C”, define meanings that we are accustomed from computer-based 
instruction. The third letter “O” refers to the word online, which means the course environment where the 
important key terms are the Internet, Web and networks for a MOOC. The final letter “C” refers to the word 
Course, which means implementing an educational plan with a pedagogical approach in MOOCs. The definition 
of the tenn course may vary in meaning as it may focus on discovering and creation as well as repeating and 
drilling. In terms of presentation of content, the course can be presented with a structured or semi-structured 
content. 

2.2 A brief history of MOOCs 

Two individuals used the tenn MOOC first: Dave Connier and Bryan Alexander (de Waard, et al. 2011; 

Hennan 2012). The real beginning of the MOOCs started in 2008 by a course called "Connectivism and 
Connective Knowledge” (CCK08) (de Waard et al., 2011a; Fini, 2009) which was facilitated by George 
Siemens and Stephen Downes (Siemens & Downes, 2008). Other connectivist MOOC examples followed 
CCK08. Connectivism and Connective Knowledge” (CCK08, CCK09, CCK1 1, CCK12), Online Learning for 
Today and Tomorrow (EduMOOC), Personal Learning Environments and Networks and Knowledge (PLENK), 
Mobile Learning (MobiMOOC), Change: Education, Learning and Technology (Changel 1), Digital Storytelling 
(dsl06), Learning and Knowledge Analytics (LAK11, LAK12), Connected Learning MOOC (clMOOC), 

MOOC MOOC: Dark Underbelly (MMDU), MOOC MOOC: Critical Pedagogy (MMCP), Rhizomatic 
Learning: The Community is the Curriculum (Rhizol4) and Rhizomatic Learning: A Practical View (Rhizol5) 
can be given as examples to other connectivist MOOCs. 

The success of connectivist MOOCs led to Stanford Al-based MOOCs. Sebastian Thrun and Peter 
Norvig used MOOC concept to teach Artificial Intelligence in 201 1. They used a system developed by Amazon 
through algorithms to assess and evaluate enrolled students (Stevens, 2013). In 2012, Udacity, Coursera and 
MIT edX were founded to provide MOOCs (Martin, 2012). Figure 1 explains birth and evaluation of MOOCs. 
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► j n (| ue nce □ Entrepreneurial initiatives 

Figure 1. 

MOOCs and Open Education Timeline (Yuan, 2015). 

2.3 Types of MOOCs. 

With growing interest and rise of MOOCs, different MOOC types appeared. Though the letters that constitute 
the acronym “MOOC” describe one basic form; in practice, there are different variations in size, openness, 
delivery mode (Diaz, Brown and Pelletier, 2013). MOOCs have some common features, but they differ on the 
learning theory and pedagogical model on which they stand (Rodriguez, 2012). They are even so distinct in 
pedagogy that it is confusing to designate them by the same term (Hill, 2012). Siemens (2012), who is one of 
the pioneers of MOOCs, categorizes them into two: cMOOCs and xMOOCs. It seems that while their MOOC 
part presents the same characteristics, the initial letters (c for connectivist and x for extended) actually define 
what type of MOOCs they are. 

cMOOCs provide great opportunities for non-traditional forms of teaching approaches and learner- 
centered pedagogy where students learn from one another (Yuan and Powell, 2013) and emphasize creation, 
creativity, autonomy, and social networked, distributed learning. In contrast to first connectivist MOOCs, 
xMOOCs embraced conventional (Weller, 2012), “drill and grill” approaches (Siemens 2012) which emphasize 
a more traditional learning approach through video presentations, short quizzes and testing. However, it should 
be noted that xMOOCs have been criticized for adopting traditional knowledge transmission models (Larry, 
2012). To sum up, cMOOCs focus on knowledge creation and generation, while xMOOCs focus on knowledge 
duplication (Siemens, 2012). Rodriguez (2012) also stated that cMOOCs belong with the connectivist distance 
education pedagogy while xMOOCs belong with the cognitive-behaviorist with some constructivist 
contributions. When compared, it is seen that traditional pedagogies and xMOOCs move linear and structure a 
hierarchical system though learning in connectivism and cMOOCs act like a network and structure a chaotic, 
connected, distributed learning ecosystem (Figure 2). 
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Connectivist networked education 


Traditional linear education 


Figure 2. 

How tratidional pedagogy and connectivism show up (Bozkurt, 2013). 

Currently, there is a growing trend to be digitalized in many areas including education. MOOCs are 
forcing pedagogy for evaluation (Diaz, Brown and Pelletier, 2013) and it is believed that present educational 
structure will not efficiently serve the needs of tomorrow (de Waard et al., 2011b). As a response to this need, 
connectivism was put forth for consideration by Siemens (2004) as a new learning theory to explain the way to 
learn in digital age on networks. Downes (2007: para.2) defines connectivism as “the thesis that knowledge is 
distributed across a network of connections, and therefore that learning consists of the ability to construct and 
traverse those networks”. Siemens (2004) states that connectivism is the integration of principles explored by 
chaos, network, and complexity and self-organization theories. 

2.4 Current Status: The Ivy League of MOOC providers 

The first MOOC launched in Canada, called cMOOC afterwards, gained a great success as a new 
phenomenon and attracted a lot of attention as a new form of delivering learning content. Following this 
success, many universities and companies rushed to be part of these “global mega classes” (Bozkurt, 2015, 
p.61). Different xMOOCs followed one another to join “the copycat rush to jump on the xMOOCs bandwagon” 
(Daniel, 2012: p.14) and the promise of MOOCs encouraged elite universities to put their courses online 
offering them for free or charging a small fee for certification on MOOC platforms (Yuan and Powell, 2013). 

The main xMOOC providers are all originated from the US: Coursera, EdX, Udacity and Khan 
Academy are main players in ivy league and have millions of students from all around the world. The European 
Association of Distance Teaching Universities (EADTU) launched OpenupEd and the Open University UK 
launched FutureLeam as a European counterpart to other MOOC platforms to connect learners from all over the 
globe. Besides, Iversity, Udemy, CourseSites, MOOC2Degree, Canvas Network, P2PU and Thinkful are other 
initiatives that had a fresh start up to play in the ivy league. 

2.5 MOOCers and Constraints in MOOCs 

One of the significant problems that MOOCs face currently is quality assurance and accreditation. On 
the other hand, assuming that they have already been offered by higher education institutions; a modular system 
might be offered in which a total of many relevant MOOCs can be credited. 

High dropout rates and behaviors of lurkers are the two important questions that most of the educators 
try to answer (Cormier & Siemens, 2010; Rodriguez, 2012) not only for MOOCs but also for many online, open 
and distance learning systems. The evidences prove that completion rates are very low when compared with the 
number of participants enrolled in MOOCs (Daniel, 2012; Jordan, 2012; Yuan and Powell, 2013) and most of 
the participants are lurking in MOOCs. 

In open courses, participants engage in different levels (Cormier & Siemens, 2010) and high active 
participation is generally a desired situation in any kind of teaching/leaming system. During the MOOCs, 
participants can choose to have an active or passive role. Hill (2013) identifies four types of learners: Lurker, 
active participants, passive participants and drop-ins. Similarly, de Waard et al. (2011b) categorize participants 
and their roles as lurking participants, moderately active participants and memorably active participants. “1%” 
or “90-9-1" rule (Nielsen, 2006) explains what happens in MOOCs like many other online communities (Figure 
3). According to that rule; 

• 90% of users are lurkers. They read or observe, but don't contribute. 


30 


• 9% of users contribute from time to time, but other priorities dominate their time. 

• 1% of users participate a lot and account for most contributions. 




10% of postings 


from 9% 
of users 



No postings 
from 90% of users 


/ 


f 


Figure 3. 

90-9-1 rule (Nielsen, 2006). 

With a more functional definition, we can categorize participants in MOOCs into four: lurkers, 
contributors, creators and drop-ins. Drop-ins are different from first three types because they can join in or drop 
out anytime according to their own learning needs and expectations. 90-9-1 rule more or less fits to this pattern 
and explains the rates in MOOCs. Therefore, we may presume that in a MOOC, approximately 90% of 
participants are lurkers, 9% contributors and 1% creators. Drop-ins can fall into any of these categories. 

The essence of the MOOCs is learners themselves. Therefore, understanding participants’ 
characteristics, preferences and experiences during the MOOCs stands as an important question to seek for. 
According to a review of the published MOOC literature between 2008 and 2012, conducted by 
Liyanagunawardena, Adams and Williams (2013), most articles published so far have dealt with empirical 
evidence from case studies, MOOCs’ influence on higher education structure, or educational theory relating to 
MOOCs. According to this review of literature, it is seen that there is a lack and a need for studies on 
MOOCers’ preferences and attitudes and this study intends to meet this need. 

3. Heutagogy: As a Theoretical Framework to Understand MOOCs and MOOCers 

In massive open online courses, what is the theoretical background in terms of MOOCers? This 
question arises another question: Who are the MOOCers? MOOCers are generally adult learners. Within this 
perspective, heutagogy explains adult learners in terms of lifelong learning, Web and Web technologies as a 
learning environment and learning tools. 

Heutagogy is a “net-centric” theory like connectivism (Anderson, 2010). Rooted from andragogy, 
heutagogy poses as an extension of pedagogy and andragogy. Heutagogy is for adult learners and “has been 
proposed as a theory for applying to emerging technologies in distance education and for guiding distance 
education practice and the ways in which distance educators develop and deliver instruction using newer 
technologies such as social media” (Blaschke, 2012: p.l). With all attributes it has, heutagogy stands as an 
appropriate theoretical background to study and understand MOOCs. 


4. Related Research 


Even though limited number of the research regarding MOOCers attitudes and preferences, some of the 
reports and articles provide some data related to research objectives of this study. University of Edinburg (2013) 
reported its six MOOC experiences on Coursera platform including some demographics of participants in these 
MOOCs. Open2Study similarly presented a report prepared by Kevat (2013). Likewise, papers and reports were 
also presented by Huhn (2013) regarding four UW-Madison’s MOOCs on Coursera, and Seaton (2013) 
regarding 16 MITx and HarvardX MOOCs. However, the data presented in these papers are limited to xMOOCs 
and present basic demographics of MOOCers. 


5. Research Objectives 


This study intends to get a descriptive picture of participants’ attitudes and preferences in a MOOC. 
With this purpose, the survey aimed to identify and learn MOOCers’; 


31 


1 . Demographics: Gender, age, educational background and occupational status, 

2. MOOC experiences: previous MOOC experiences of participants, time spent for the MOOCs, 

3. Reasons to enroll in the MOOCs, 

4. Preferences: Communication/leaming environments, content types, evaluation formats and interaction 
levels, 

5. Difficulties they experienced and their willingness for future MOOCs, 

6. Attitudes during and after a MOOC. 

6. Methodology 

6.1 Participants 

E-Learning and Digital Culture MOOC (EDCMOOC), initiated on January 28th, 2013, was held in 
Coursera platform for five weeks. A total of 42874 participants enrolled in the course from all around the world 
(Figure 4). Even though the platform was Coursera in which usually xMOOCs were implemented, EDCMOOC 
was a bit constructivist and connectivist in its nature. EDCMOOC as a hybrid MOOC (Waite, Mackness, 
Roberts, and Lovegrove, 2013) that stimulating a liminal state (Perkins, 2006) for many learners and perhaps 
ultimately leading to a way of thinking about digital culture that was previously unavailable (Sinclair, 2013). 
EDCMOOC used different platforms such as Facebook group, Google Plus, Google Hangout, Twitter, YouTube 
and other Web 2.0 tools of which generally used in cMOOCs and an LMS style learning platform of which 
generally used in xMOOCs. Participants of this research were from a Facebook Group 1 which is a social 
network platform that EDCMOOC ers used. 



Figure 4. 

EDCMOOC Crowd Map 2 

6.2 Instrumentation and Process 

Considering the limited time period to reach participants, survey technique was used to explore the 
research questions. For this research, a 13-item online questionnaire was constructed and also a final open-ended 
question was added to learn more about their insights, thoughts and reflections. 

Convenience sampling was used for the research and the survey was conducted in Facebook Group. The survey 
was prepared in Google Drive and the link was shared for various times in the Facebook Group between 


1 https://www.facebook.com/groups/edcmooc/ 

2 http://edcmoocteam.wordpress.com/tag/statistics/ 
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February -April 2013. The participation to the survey was voluntary and 161 EDCMOOCers completed the 
questionnaire. 

6.3 Data Analysis 

In this study, both quantitative and qualitative data were analyzed and interpreted using descriptive 
analysis. A software package for statistical analysis was used in order to analyze quantitative data gathered from 
questionnaire items. Percentage, as a relative standing value, was calculated for each item and then findings 
were interpreted. For qualitative data, a word (tag) cloud was created to be able to get a weighted list in a visual 
representation for text data gathered from the open ended question. 

6.4 Strength and Limitations 

This study has some strengths and limitations. The strengths of this study are as following: This study 
reveals characteristics of MOOCers in a Hybrid MOOC which is an emerging third type of MOOC. Besides, 
there are limited number of studies that focus on MOOCers and as a research that focus on Hybrid MOOCers, 
this study intends to contribute to relevant literature. However, this study has some limitations. First of all, this 
study has descriptive results which seeks to answer “what” question. On the other hand, authors of this study 
believe that further research are needed to answer “how” and “why” questions for MOOCers’ attitudes and 
preferences. 


7. Findings 


7.1 Who are the MOOCers? 

Table 1 presents characteristics of the MOOCers. According to the survey results, 33,5% of the 
participants were male and 66,5% were female. The findings about age and education indicate an important 
point about MOOCers: 81,1% of the MOOCers, who were probably out of conventional education, were above 
30+ years old. This finding shows that MOOCs are rising star of lifelong learners and they use MOOCs as an 
extension of their conventional education. Another significant finding is about their educational background. 
62,8% of the MOOCers have master or doctoral degree. The final finding regarding MOOCers’ demographic 
characteristics is about their occupational status. A total of the 88,8% of MOOCers are either working, or 
working and attending their education at the same time. All in all, considering that these people enroll these 
courses voluntarily, the findings about age, education and occupation status show that the majority of the 
MOOCers participate in these courses as a lifelong learning activity. 


Table 1. 

The Characteristics of the MOOCers 


Gender 

• Male (33,5%) 

• Female (66,5%) 

Age 

• 18-29(19,9%) 


• 30-39 (32,9%) 


• 40-49 (16,8%) 


• 50-59 (23,6%) 


• 60+ (6.8%) 

Education 

• High school (2,5%) 


• College/Associate degree (8,1%) 


• Bachelors degree (26,7%) 


• Master degree (52,2%) 


• Doctorate degree (10,6%) 

Occupation 

• I am a student ( 1 1 ,2%) 


• I am working (53,4%) 


• I am a student and I am working (35,4%) 


These findings have some similar patterns in tenns of age and education. According to University of 
Edinburg (2013), participants from all age categories, with the highest proportion aged 18-24 years old (21%), 
25-34 years old (33%) and 35-44 years old (18%). In tenns of education level, Coursera reported that of all 
Coursera participants, 18,1% have high school degree, 5,2% have associate degree, 38,9% have bachelors 
degree, 37,8% have postgraduate degree. When compared these xMOOCs and Hybrid MOOC data, though age 
variable seems to have a similar pattern both in x and Hybrid MOOCs, education level of MOOCers in Hybrid 
MOOCs has a higher tendency. These findings reveal that most of the MOOCers are middle aged both in x or 
Hybrid MOOCs and most of the MOOCers completed highest level of education. 
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7.2 MOOCers’ Experiences 

Table 2 presents participants’ previous MOOC experiences and the time spent for the MOOC. 71,4% 
of the MOOCers joined this MOOC for the first time and 28,6% had previous MOOC experience. Majority of 
the MOOCers spent 3-6 hours in a week for the MOOC. 


Table 2. 

Participants Previous MOOC Experiences and Time Spent During a MOOC 


First MOOC 

• Yes 

(71,4%) 

experience 

• No (28,6%) 

Time Spent 

• 1-2 

(19,9%) 


• 3-4 

(31,1%) 


• 5-6 

(26,1%) 


• 7-10 

(14,9%) 


• 11 + 

(8,1%) 


According to University of Edinburgs report, 75,1% of MOOCers did not participated in MOOCs 
previously and 24,1% had previous MOOC experience. These findings confirm each other and indicate that 
interest against MOOCs will go on for a while considering the most of the MOOCers are new participants and 
they continue to take new MOOCs. 

7.3 Reasons to enroll 

Table 3 presents the reasons to enroll in the MOOCs. Participants were permitted to choose as many 
appropriate options as they find relevant regarding to their reasons to enroll. The findings are ranked according 
to their frequency. 65,8% of the MOOCers join in these mega classes as they are free. 65,2% of the MOOCers 
attend these courses for professional development and 60,2% for personal development. The top three reasons 
indicate once again that a big majority of the MOOCers perceive MOOCs as an opportunity for lifelong 
learning. 54,7% of the MOOCers prefer e-leaming, so it stands as the fourth most important reason. Following 
that, 51,6% of the MOOCers indicated that they wanted to experience MOOCs and 49,7% of them enrolled for 
curiosity. 37,3% of the MOOCers stated that the MOOC was about their subject area that they study or work. 

3 1,7% of the MOOCers joined the MOOC to get a certificate. 23,6% of the MOOCers stated that they enrolled 
the MOOC as it was offered by a credential institute. 14,9% of the MOOCers stated that they enrolled in MOOC 
as they were unable to get the course in their residential area. 


Table 3. 

Reasons to Enroll in MOOCs 


Reasons • 

It is free (65,8%) 

• 

For professional development (65,2%) 

• 

For personal development (60,2%) 

• 

I like e-leaming (54,7%) 

• 

I wanted to experience MOOC (51,6%) 

• 

Curiosity (49,7%) 

• 

This MOOC is about my subject area (37,3%) 

• 

To get a certificate (3 1,7%) 

• 

Course is offered by a credential institute (23,6%) 

• 

I am unable to get this course in my city/region/country (14,9%) 


7.4 MOOCers’ Preferences about the Course Design 

Table 4 presents the findings regarding MOOCers’ preferences about communication/leaming 
environments, content structures, interaction level and evaluation type. Participants of this survey are allowed to 
select as many appropriate options as possible for this questionnaire item. A great majority of the MOOCers, 
83,2%, prefer official MOOC Web page. 67,1% of the MOOCers prefer social networks such as Facebook, 
Twitter or Google Plus. Multimedia presentation environments with 52,8% and virtual classes with 42,9% come 
after. 31,7% of the MOOCers prefer forums, 27,3% prefer blogs and 9,3% prefer wikis. 

The following item was about MOOCers’ preferences about structures. 15% of the MOOCers tend to be passive 
and they prefer structured content and minimum effort to obtain the learning content. On the other hand, 33,5% 
of the MOOCers prefer structured content, additional sources and discussing with others. 50,9% of the 
MOOCers prefer semi-structured content, exploring, discussing and creating. 
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The item about interaction level was designed whether MOOCers are lurkers, contributors or creators. 
24,8% of the MOOCers like reading the content, watching videos and not to communicate with others. These 
features represent lurkers. 31,1% of the MOOCers like rating the topics (like, favorite, share etc.) and 
commenting if they need to. These MOOCers may be called as contributors who are generally moderately 
active. 44,1% of the MOOCers like commenting on a lot of topics, discussing, presenting new ideas and meeting 
with different people. The final group might be called as creators who are always active and seeking every 
single opportunity to interact with other participants and the content. It is seen that in contrast to general 
assumption, in this hybrid MOOC, MOOCers tend to be in reverse categories presented in 90-9-1 rule. 

The final item was about MOOCers’ preferences about evaluation type. 8,7% of the MOOCers want to 
be evaluated by peers, 10,6% want to be formally evaluated (tests and quizzes) and the majority with 63,4% 
want to be evaluated both formally and by peers. Those who believe that evaluation is not important are 17,4 
percent. 

Hybrid MOOCers preferences about communication/leaming environments, content structures, 
interaction level and evaluation formats strongly related to each other. It seems that they are generally self- 
determined, they like active participation and high interaction both with content and other learners. They prefer 
traversing on networks using different platfonns and Web 2.0 tools and services. There is an increasing 
tendency in structure of the content and and interaction level from behaviorist-cognitive pedagogies to 
constructivist-connectivist pedagogies. The preferences of Hybrid MOOCers also reveal its connection with 
heutagogy since heutagogical approach recognizes the need to be flexible in the learning process and let the 
learners design the actual course they might take by negotiating the learning. 


Table 4. 

Preferences of MOOCers 

Communication/ 

• Official MOOC Web page (83 ,2%) 

Learning environments 

• Social Networks ( Facebook, Twitter, Google plus etc. ) (67,1%) 

• Multimedia presentation environments (You Tube, Prezi, Flickr etc.) 

(52,8%) 

• Virtual classes (Google hangout, WizIQ, Blackboard etc.) (42,9%) 

• Forums (31,7%) 

• Blogs (27,3%) 

• Wikis (9,3%) 

Content structures that 

• I like structured content, reading and watching (15,5%) 

they prefer 

• I like structured content, additional sources and discussing with others 
(33,5%) 

• I like semi-structured content, exploring, discussing and creating (50,9%) 

Interaction level, 

MOOCers like: 

• Reading the content (texts, comments etc.), watching videos, not to 
communicate with others (24,8%) 

• Rating the topics (like, favorite, share etc.), commenting if have to (31,1%) 

• Commenting on a lot of topics, discussing, presenting new ideas and meeting 
with different people, (44,1%) 

Evaluation fonnats 

• To be evaluated by peers (8,7%) 

• To be formally evaluated (tests and quizzes) (10,6%) 

• Both (to be formally evaluated and to be evaluated by peers) (63,4%) 

• Being evaluated is not important (17,4%) 


7.5 Problems MOOCers Encounter in MOOCs 

Table 5 focuses on one of the important points: Difficulties that MOOCers experienced. According to survey 
results, the main difficulty that MOOCers encounter is time issues. As most of the MOOCers are working 
(Table 1.), time management and allocating free time stand as the biggest problem with 51,6%. Following 
different platforms all at once is also a big difficulty for the MOOCers. Another problem that frequently 
mentioned is the chaotic atmosphere at the beginning of and during the MOOCs in forums. Digital literacy with 
5,6% and content with 5,6% are other reported difficulties. 21,7% of the MOOCers stated that they didn’t 
experience any difficulty during the MOOCs. 

Considering their overall experiences, MOOCers were asked if they planned to enroll in another 
MOOC. A great majority of MOOCers, 91,3%, expressed that they planned to enroll in other MOOCs, while 
only 1,2% stated that they wouldn’t enroll in another MOOC. 7,5% of the MOOCers said that they would 
probably enroll in other MOOCs. A great majority of the MOOCers’ intention to join other MOOCs indicates 
that future MOOCs will call attention for a long time. 
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Table 5. 

Reasons to Enroll in the MOOCs 


Difficulties MOOCers 
experienced 

• Time issues (51,6%) 

• Following different platforms (39,8%) 

• Digital literacy (need to use different Web tools) (15,5%) 

• Language (English as a second language) (5,6%) 

• Content (5,6%) 

• None (21,7%) 

Willingness to enroll 

• Yes (91,3%) 

for future MOOCs 

• No (1,2%) 


• Maybe (7,5%) 


7.6 Attitudes of MOOCers during and after a MOOC 

In the final part of the questionnaire, MOOCers answered a final question to state their insights, thoughts and 
reflections. Some of these comments are provided in following part: 

• A great experience! A trip into the world of knowledge! 

• Memorable learning experience happens when you learn, create, and share relevant things. 

• They make me feel badly for all the time wasted in traditional education 

• I love them! I might get sick of them, but hoping to enjoy the experience whilst my enthusiasm is 
there. My knowledge and skill base has improved greatly through the 3 MOOCs I have 
participated in. 

• Seemed like an endless amount of learning was available. I truly enjoyed the entire experience. 

• The MOOC I did, on e-leaming and digital cultures was well worth the effort. I enjoyed it 
immensely. 

• I have loved the way its messiness mirrors the chaos which characterizes the information flow of 
the Net. 

• I love the collaboration of the students in the groups. They made all the difference for me. 

• For me were the best, the minimalize the social media platforms. I'm always afraid of losing 
something interesting if I'm not on all social media channel. But this themes are sooo interesting 
that I must be all time on internet! ! ! Thanks for this MOOC, it was amazing and it is a very 
important part of my life. 

• MOOCs are a new trend to be considered in the education scenario of the 21st century. They 
provide more flexibility and adaptation to the learner's profile. I believe this open education hype is 
here to stay. 

Figure 5 presents a visualization of word frequency as a weighted list based on the comments of the 
MOOCers. These comments clearly indicate that MOOCers love the way the MOOCs are and EDCMOOC was 
a great experience for them. 
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Figure 5. 

A word cloud about MOOCers attitudes and feelings about MOOCs. 

Along with the positive comments, some participants reflected negative thoughts in their comments. 
The following comments explain the difficulties that MOOCers experienced: 

• There were too many forums around the same topic to be able to keep track. The MOOCers should 
build topic wise forums so that we have discussion at same place. Also instead of getting onto too 
many platforms (Google+, Facebook, YouTube), we could have just use Coursera platform or one 
more. They had mandated to upload pics at Flickr, so I had to create a separate ID for that. I think 
such things should be refrained from. 

• I just need a better method to process and interiorize this information tsunami. 

• Too many people in the class. Need more interaction, accessibility with instructors. Liked the 
content but had hoped for applications that are more practical. 

• I think I would have been more active in commenting if the forums were more user friendly and 
easier to navigate. Perhaps a phpBB or VBulletin board would work better. I don't use Facebook, 
or Twitter. 

• Interaction with the other students and the teachers is very important to my learning and 
motivation. Just leaving posts is not enough for me. I like real-time exchanges and have taken one 
class before that was much better in this facet. 

• I now realize that a MOOC takes more time than covering the content and assignments. To add 
dimension and connection to the experience, it's important to have the time to be part of a smaller 
cohort or to join multiple conversations in the hopes of getting a response. 

Figure 6 presents a visual representation for text data about the difficulties of the MOOCers. The 
problems they faced were about different problems such as chaotic atmosphere in forums, lack of interaction 
among learners and following different platforms. 
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Figure 6. 

A word cloud about MOOCers difficulties during the MOOC 

8. Discussion and Conclusion 

This study reveals some important findings about preferences and attitudes of MOOCers. MOOCs have 
a great popularity and participants’ preferences demonstrate that they are a widely preferred lifelong learning 
opportunity. Majority of the MOOCers are middle aged, working and have an MA or PhD degree. Their primary 
difficulty is time management. Most of the MOOCers prefer high engagement level and want to be evaluated by 
peers and formal automated evaluation systems. They prefer familiar platforms such as official MOOC page or 
social networks. 

Considering the reasons why MOOCers enroll, it is clear that that they are lifelong learners and 
majority of them join in MOOCs for learning rather than getting a certificate or as they are offered by a 
credential institute. The top reasons to choose MOOCs as a learning environment generally emerge from 
intrinsic factors rather than extrinsic ones. Based on these findings, it might be said that MOOCs are a 
worthwhile option to augment learning beyond the walls of higher education. 

Another important point is the presentation of the MOOC contents, and MOOCers’ 
communication/leaming environment preferences. In these massive courses, participants’ characteristics, needs 
and preferences regarding to learning environments, content structures, interaction level and evaluation format 
vary. Therefore, while designing a MOOC, MOOCers’ diversity should be taken into account. 

The biggest problem MOOCers experience is time issues. This is probably related to their occupational 
status. Therefore, the time required from a MOOCer, the length of the MOOCs and the cover of the MOOC 
content need to be structured carefully. 

Some of the answers to open ended questions indicate that the chaotic environment that MOOCers 
experience is an important obstacle. It is obvious that not all MOOCers are autonomous and they need more 
support. With this in mind, learner support systems such as course orientation and guides should be prepared 
because most of the newbie MOOCers are not experienced MOOC participants. Knowing what will happen in a 
safe environment might also lessen the high drop-out rates that is observed in many MOOCs. 

MOOCers also express difficulty of following and logging in different platforms. It is clear that 
aggregation services are needed as MOOCers suffer from tracking and using different platforms at the same 
time. User friendly interfaces need to be designed and integration with social networks and other services need 
to be considered to ease these difficulties and to enrich the flow of information by providing different Web 
platforms and services. 

Considering high drop out and low retention rates, MOOC learning process can be designed in a more 
engaging way. A MOOC designed with gamification elements may be helpful to increase learning motivation, 
and to lessen high drop-out rates. Leaderboards, badges, experience points and other gamification elements may 
motivate those who represent lurkers with 90% as well as contributors with %9 and creators with %1. So, future 
research can be conducted in MOOCs designed with gamification principles. 
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The overall picture demonstrate that Hybrid MOOCers are self-determined and prefer a flexible 
structure for learning. In Hybrid MOOCs, MOOCers are able to manage their own learning and emphasis is 
placed on development of learner capacity, capability and the process all of which are core of the heutagogy as a 
continuum of pedagogy and andragogy. 

Based on the arguments above, we define MOOCs as mega classes and Hybrid MOOCs as networked 
learning spaces in which behaviorist, cognitive, constructivist and then connectivist pedagogies are applied and 
MOOCers, in their learning quest, traverse and cross-pollinate among multiple paths and layers of hybrid 
learning ecologies. The flexible nature of Hybrid MOOCs allows MOOCers to learn what they need and also 
allows them to what extend they progress. 

In conclusion, most of the MOOCers seem to be satisfied with their MOOC experience and have 
positive impressions about MOOCs. A great majority of MOOCers plan to join another MOOC in future and it 
proves that MOOC myth will go on for a long time. On the other hand, future research are needed to improve 
MOOCs and to understand MOOC participants. However, it must be also noted that these findings are derived 
from a hybrid MOOC, thus similar research are needed for c and x MOOCs. As a final remark, it is clear that 
more hybrid MOOCs and more research regarding these MOOCs are needed to be able to understand these 
hybrid MOOCs and to develop a hybrid MOOC model. 

Regarding the results of this study, the following implications can be taken into consideration for future 

research: 

• In addition to extended (xMOOCs) and connectivist (cMOOCs) pedagogy, future research are needed 
to understand hybrid pedagogy and to be able to meet Hybrid MOOCers’ needs. 

• Within a learner-centric view, further research is needed to explain c and x MOOCers’ attitudes and 
preferences similar to this study that focused on hybrid MOOCers. 

• Type of learners may vary in a MOOC. Future research that focus on type of learners (Lurkers, 
contributors, creators and drop-ins) and their activities in a MOOC may help instructional designers to 
design more effective and efficient MOOCs. 
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Abstract 

The study investigated whether the accent of the narrator in a multimedia tutorial about money 
management affected participants’ learning and attitudes toward the narrator. Sixty-five Chinese students at a 
Midwestern university in the United States were randomly assigned to one of two groups in this experimental 
design. One group heard an American narrator speaking English in the tutorial, and the other group heard a Chinese 
narrator speaking English. Data to test the dependent variables were collected through a learning achievement test 
including both recall and above-recall level questions and an attitude survey. Data analyses revealed that there was 
no significant difference in overall learning as well as in recall level learning and above -recall level learning 
between the two accent groups. Elowever, the group of Chinese students who heard the narration spoken with the 
American English accent had significantly more positive attitudes toward their narrator than the group of Chinese 
students who heard the narration spoken with a Chinese accent. 

Keywords: voice principle, multimedia learning, accent, non-native speaker, learning, attitudes 

Introduction 

Research in the field of multimedia learning has yielded principles for the design of effective multimedia 
instructional messages including Mayer’s (2005) principles regarding voice. According to the voice principle, 
students learn more deeply when the narration in a multimedia lesson is spoken by a native voice rather than a non- 
native voice. Voice attributes include, but are not limited to, gender, age, pitch, volume, pace, and accent. As Mayer 
described native and non-native, it can be concluded that the voice principle includes accent. 

Mayer, Sobko, and Mautone (2003) found that an unusual accent, which was identified as a foreign accent 
in their study, may create more extraneous cognitive load for the students. Under the theory of cognitive load, Mayer 
and his colleagues also assumed that performance during knowledge acquisition depends on the cognitive resources 
available for information processing. As a result, when learners use more cognitive resources trying to understand an 
unfamiliar accent, they have less cognitive resources available to process the information. Their performance may 
not be as good as that of learners who interact with a native accent. However, it is a different question to ask if the 
voice principle applies to non-native tutorial users who share their first language with a non-native speaker. For 
example, does the voice principle apply when an English narration is not in the native language of the speaker or the 
tutorial users, and the speaker and the users share the same first language? In this case, the speaker and the tutorial 
users speak English with a shared foreign accent. The accent of the speaker, though classified as foreign or non- 
native, is not unusual to the non-native tutorial users. Therefore, in this situation the voice principle cannot be used 
to predict performance of non-native users. Because the generalizability of the voice principle beyond native 
speakers is unknown, there is a rationale to investigate the effects of accent on non-native speakers. 

According to Crystal (2003), only one out of every four users of English in the world is a native speaker of 
the language, and the vast majority of verbal exchanges in English do not involve any native speakers of the 
language at all. Interestingly, in most cases, English is often “a ‘contact language’ between persons who share 
neither a common native tongue nor a common (national) culture, and for whom English is the chosen foreign 
language of communication” (Firth, 1996, p. 240). Because of the large number of non-native English users, 
researchers should consider these users in their studies of instructional strategies and systems. 

As more and more people speak English as a second language, there are more and more English accents. 
Adjusting to and accommodating various accents has become an essential ability for effective and respectfid 
communication (Cheng, 1999). A question we now face in education is, “How might accents impact a student’s 
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learning?” In particular, given the rapid expansion of multimedia instruction, “What are the effects of accented 
narration in multimedia instruction on the learning of second language learners?” Note that the interest in this study 
is non-native students learning academic content in English, not their learning of English. 

Although there are numerous design and implementation considerations involved in implementing 
multimedia instruction, the focus of this study is on the accent of the speaker in computer-based tutorials because of 
the popularity of multimedia tutorials, especially in online learning. Multimedia instruction can be produced by 
institutions or individuals. “Homemade” tutorials can be produced by teachers or trainers, for example, by using 
screencasting software to add narration to PowerPoint presentations subsequently posted online. This study results 
inform instructional designers and teachers how to select, design, and implement tutorials most effectively based on 
the speaker’s accent. Specifically, the study investigates effects of the English speaker’s voice in a tutorial, across 
two different accents: native American-accented English and non-native shared Chinese-accented English. A 
multimedia tutorial regarding money management was used to investigate the effects of the speaker’s accent on 
participants’ learning and on their attitudes toward the speakers. The research questions addressed by the research 
were: 

1. Does tutorial narrator accent (native American-accented English versus non-native shared Chinese- 
accented English) differentially affect learning? 

2. Does tutorial narrator accent (native American-accented English versus non-native shared Chinese- 
accented English) differentially affect attitudes toward the narrator? 

Literature Review 


Voice Principle 

Mayer (2005) has articulated and investigated the voice principle for the design of multimedia instructional 
messages. According to the voice principle, people assumed to be native speakers learn more deeply when the words 
in a multimedia module are spoken by a native-accented human voice speaking their native language rather than a 
foreign-accented human voice or a machine voice (Atkinson, Mayer, & Merrill, 2005; Mayer, Sobko, & Mautone, 
2003). Mayer et al. (2003) conducted an experiment to examine the idea that the speaker’s voice in multimedia 
lessons carries important social cues that can influence the process and outcome of leaning. The narrator’s voice in 
the tutorial was either a native speaker of American English or a non-native English speaker with a Russian accent, 
i.e., native-accented speech vs. foreign-accented speech. In the experiment, learners who were seated at a computer 
workstation received a narrated animation about lightning formation. They then took a retention test, took a transfer 
test, and finally completed a speaker-rating survey. Overall, there was a voice effect, in which the native human 
voice group learned more and was better able to apply what was learned to solve new problems. The participants in 
the native accented group scored better on the learning transfer test than the participants in the foreign accented 
condition, resulting in a Cohen’s d statistic of .80 (a large effect). However, the students who received a narrated 
animation with the Russian-accented voice performed as well on the retention test as the students who received a 
narrated animation with a native American accent. Learners in the two accent groups made different social 
judgments about their respective speakers. The participants who listened to the native American-accented voice 
rated the narrator more positively than the participants in the other group rated the speaker with the foreign accent. 

The study, however, did not provide clear information on the criteria for choosing the participants. The 
researchers did not mention if American English was the first language of the participants or if these students could 
speak a language other than English. It is not known whether the study would yield the same results if the students 
spoke a language other than American English as their first language, in particular if the students’ first language was 
Russian. 

It is possible that Mayer (2009) recognized the limitation of the literature addressing the voice principle. 
Thus, he considered the voice principle to be in its preliminary stage and called for additional experiments. In 
particular, Mayer cited the work of Nass and Brave (2005) to recommend future research to investigate how the 
effects of voice cues in multimedia instructional messages may be different for different kinds of learners. 
Particularly, future research was recommended to figure out whether people learn better when they perceive that the 
instructor’s voice comes from someone like themselves. Nass and Brave (2005) also suggested that people may be 
more convinced by online spoken messages when they perceive the speaker’s voice to be coming from someone like 
themselves in terms of gender, race, ethnicity, or emotional state. 

Cognitive Load Theory 

The voice principle can be explained from the viewpoint of cognitive load theory (Chandler & Sweller, 
1991; Sweller, 1998). Cognitive load theory maintains that our working memory is limited with respect to the 
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amount of information it can hold and the number of operations it can perform on that information (Van Gerven, 
Paas, van Merrienboer, Hendriks, & Schmidt, 2003). That means a learner should be encouraged to use his or her 
limited working memory efficiently, especially when learning a difficult task (Van Gerven et ah, 2003). Thus, 
instructional designers need to find ways to help optimize the working memory by developing quality instruction 
and limiting extraneous cognitive load, potentially including accents of narration, which can distract learners. 

Mayer et al. (2003) pointed out that when students process a human voice speaking with a native accent, 
they use fewer cognitive resources than when they listen to a human voice speaking with a foreign accent or a 
machine-synthesized voice. Therefore, more cognitive resources are available for students to deep process the 
instructional message when they are listening to a native speaker. The extraneous cognitive load results because the 
accents have not been incorporated into the students’ prior knowledge. When trying to process words spoken in a 
foreign accent, students allocate more time in understanding words separately, rather than processing the 
relationships of the words in the sentence as a whole. Listeners may miss subsequent words while trying to figure 
out earlier words. Cognitive load theory predicts better performance on a transfer test for learners who were 
instructed by a native-accented voice than for learners who listened to a machine voice or a foreign-accented voice, 
as found in Mayer et al. (2003). However, cognitive load theory does not make any predictions concerning learners’ 
attitudes toward the speakers. 

Method 

This study investigated the effect of speaker’s accent in a money management tutorial on non-native 
English students’ learning and attitudes toward the speakers. The instructional material was a multimedia tutorial 
created by Nancy Woinoski of Pinched Head and featured on Articulate Community Showcase (Articulate, 2013). 
The tutorial originally featured English narration by a female native speaker of English. For purposes of the current 
study, the original narrator was replaced by comparable narrators speaking in English with different accents. One 
version featured a non-professional, male announcer with a native American English accent. The other version was 
re-recorded by a similarly non-professional male announcer speaking Chinese-accented English. 

A quantitative design was used to explore the research questions. The quantitative design of the study falls 
into the category of experimental research since each participant was randomly assigned to the native American- 
accented English (AAE) group or non-native shared Chinese-accented English (CAE) group, and one variable 
(English accent) was manipulated to determine its effect on the two dependent variables of participants’ learning and 
their attitudes toward the speakers (Isaac & Michael, 1995). 

Sixty-five Chinese students at a mid-sized university in the Midwest of the United States participated in the 
study. Each participant was randomly assigned to the AAE or CAE condition. After answering the demographics 
questionnaire, the participants assigned to the AAE group watched the tutorial that featured an American accent 
while the participants who were assigned to the CAE group watched the tutorial with a Chinese accent. Participants 
then took a learning achievement test consisting of multiple-choice questions of two levels: recall and above-recall. 
Finally, they completed an attitude survey including 15 speaker-rating items. The 15-item instrument was used in 
Mayer et al. (2003). Mayer et al. adapted the instrument from the Speech Evaluation Instrument by Zahn and 
Hopper (1985). 


Analyses and Results 

Thirty-three students served in the American-accented English (AAE) group and thirty-two in the Chinese- 
accented English (CAE) group. Both treatment groups were equivalent in their mean English proficiency scores. 
Data analysis revealed that the narrator accent did not have significant effects on the participants’ overall learning, t 
(63) = 0.54, p = .59, d = 0.13. Because the learning achievement test consisted of recall level and above -recall level 
items, further analyses were conducted to identify if the two treatment groups scored significantly differently on 
these two subtests. Data analysis of the independent-samples t-test yielded no significant differences between the 
two accent groups in regard to their recall level learning, t (63) = 0.65, p = .52, d = 0. 1 7. The narrator’s accent did 
not affect the participants’ scores on the above-recall subtest, t (63) = 0.24, p = .81, d= 0.06 (see Table 1). 


Table 1. Descriptive Statistics for the Learning Achievement Test 



American-accented English (N =33) 

Chinese-accented English(N =32) 

Overall learning 

M= 16.36 (62.92%) 

SD = 4.33 

M= 15.81 (60.81%) 

SD = 3.90 

Recall subtest 

M = 8.73 (67.15%) 

SD = 2.59 

M = 8.31 (63.92%) 

SD = 2.50 

Above-recall subtest 

M = 7.64 (58.77%) 

SD = 2.43 

M = 7.50 (57.69%) 

SD = 2.06 
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Regarding the participants’ attitudes toward their respective narrators, the participants in the AAE group 
students had significantly more positive attitudes toward their narrator than did participants in the CAE group, t (63) 
= 2.06, p = .04, d = 0.51. The two groups did not differ significantly in their ratings of their respective narrators on 
the separate subscales of Superiority (t (63) = 1.59,/? = .12, d = 0.39), Attractiveness (t(63) = 1.43, p = .16, d = 
0.35), and Dynamism (t (63) = 1.90,/? = .06, d = 0.47). For each individual subscale, the effect size was medium or 
close to medium. Table 2 below reports the descriptive statistics of the attitude survey. 


Table 2. Descriptive Statistics for the Attitude Survey 


American-accented English (N =33) Chinese-accented English(N =32) 


Overall attitude 
Superiority subscale 
Attractiveness subscale 
Dynamism subscale 


M = 5.87 (Max ; 
M = 6.14 (Max : 
M = 5.76 (Max : 
M = 5.67 (Max - 


: 7.50) 
8 . 00 ) 
8 . 00 ) 
8 . 00 ) 


SD = 0.92 
SD= 1.16 
SD= 1.38 
SD= 1.16 


M = 5.39 (Max = 
M = 5.71 (Max : 
M = 5.30 (Max = 
M = 5.12 (Max ; 


7.36) 

8 . 00 ) 

8 . 00 ) 

7.25) 


SD = 0.97 
SD= 1.03 
SD= 1.22 
SD= 1.20 


Discussion and Conclusions 

Effects of Accent 

The results suggest that the non-native shared Chinese English accent did not cause extraneous cognitive 
load to the tutorial learners. Such extraneous cognitive load could be a concern for designers of tutorials who are 
attempting to apply Mayer’s (2005) voice principle for native speakers of English. The target audience of the study 
was non-native English speakers while the voice principle by Mayer (2005) applies to native speakers of English. 
Therefore, the findings of the study do not directly contradict the results reported in Mayer et al. (2003) that native 
learners learned more deeply when the narration in a multimedia lesson was spoken by a native voice rather than a 
non-native voice. In fact, the study qualifies the voice principle by focusing on non-native English speakers and 
supports the conclusion that speaker’s Chinese accent does not affect overall learning and recall level learning 
among Chinese participants who shared the speaker’s accent but does affect their attitudes toward such speakers. 

The lack of significant difference between the two accent groups in their learning outcomes may be the two 
narrators were both intelligible for the Chinese students. The study participants passed the English requirements at 
the university where they were enrolled. In addition, on average, they had been in the United States for more than 
one and a half years and they had been enrolled in four semesters in the United States. Therefore, the study 
participants in the AAE group were used to the American English accent. It is likely that the participants in the CAE 
group spoke English with a Chinese accent, and they were familiar with the accent of the speaker. As a result, they 
did not have difficulty understanding their narrator. 

In order to further investigate the observed difference in learners’ attitudes toward the speakers based on 
accent, the researcher analyzed individual aspects of the attitude survey. The t-tests for the individual subscales - 
Superiority, Attractiveness, or Dynamism - did not show statistical significance for the mean difference between the 
accent groups. However, the effect sizes were close to medium he analysis of the three subscales. The failure to 
reach statistical significance might be due to the small sample size of the study, resulting in Type II error. 

Implications 

The study helped to establish the limit of the voice principle’s (Mayer et al., 2003) generalizability by 
including non-native English speaking learners. According to the voice principle, native English speakers’ deep 
learning will be significantly better when the narration is spoken with a native English accent than with a foreign 
one (Mayer, 2005). However, for Chinese speakers, a shared Chinese accent in multimedia instruction will bring 
about the same overall learning (measured by a test requiring both recall and above-recall cognition) and recall level 
learning specifically as a native English accent. While Chinese speakers learning in English appear to have a better 
attitude toward narrators with a native English accent, they do not learn better from such a narrator. 

The findings of this study help instructional designers make decisions regarding which accent to use in 
multimedia instruction for Chinese learners. Since there is no significant difference in Chinese users’ overall 
learning and recall level learning regardless of native English or shared Chinese accent, instructional designers can 
simply utilize an intelligible Chinese narrator unless attitude toward the speaker is considered important. For such 
learners, multimedia instruction with a shared Chinese accent does not cause extraneous cognitive load. With the 
popularity of e-Learning and self-made multimedia instruction, the study helps assure Chinese instructors that they 
can use their own voices to record the English narration. Providing that narrators are intelligible, Chinese students 
will learn from tutorials with a Chinese-accented English narration as much as from a tutorial voiced with a native 
English accent. 
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Limitations and Recommendations 

The results of the study might be limited only to native Chinese speakers who listen to English with a 
Chinese accent. In the non-native shared accent group, the narrator and participants spoke Chinese as their first 
language. The narrator spoke English with a Chinese accent, and it was assumed that the participants spoke English 
with a shared Chinese accent. Therefore, the results of the study cannot be applied to a situation in which the first 
languages of the narrator and tutorial users are from other cultures such as Japan or Mexico, rather than China. 

In addition, the Chinese participants in the Chinese-accented English group and their respective narrator 
shared the same first language: Chinese. In reality non-native speakers of English communicate with other non- 
native English speakers with different first languages. The researcher suggests a study of non-native speakers 
wherein the narrator and participants do not share the same mother tongue. For example, participants are Chinese 
and the narrator is Korean or Mexican. Future research, then, might compare the effects of non-native accent on the 
learning of non-native participants who share and non-native participants who do not share the narrator’s first 
language. Such a study could ascertain if a non-native, shared accent brings about better learning and/or more 
positive attitudes toward the narrator than a non-native, non-shared accent. A series of such studies can determine 
the generalizability of accent effects on learning and attitudes toward the narrator among non-native learners. 

The narrators in the study were controlled for intelligibility. The Chinese speaker in the study was easily 
understood by the Chinese students. Further research should use a heavily-accented narrator or involve two narrators 
with different levels of intelligibility. Such studies can reveal if differences in intelligibility of non-native accents 
differentially affect participants’ learning and attitudes. 
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Abstract 

This paper analyzes the implementation of an active learning strategy geared toward the gap between 
successful students’ performance on multiple choices clicker questions and their application to a given case study. 
The decision was to redesign one three-week module with the highest potential to address the identified gap in 
students’ learning. A combination of blended learning and flipped classroom strategies was used. The analysis of 
student performance showed a beneficial impact of active learning. 

Motivation and Goals of the Study 

Well designed and implemented active learning activities can stimulate overt behaviors that result in 
changes in knowledge base and skills at the higher taxonomical levels of learning outcomes (e.g. Chi & Wylie, 
2014). The focal Pharmacy Doctor (PharmD) program analyzed for this study is a highly competitive professional 
graduate program using a mastery learning instructional structure. A one-semester initial monitoring of a first 
professional year (PI) pathophysiology course revealed a potential high-impact starting point in reviewing the 
nature of instructional activities in the course. That is, the instructor realized that students’ successful performance 
on multiple-choices administered both as “clicker” questions and an in-class short quiz did not translate into the 
application of that content into a case -based SOAP note activity, a synthesis tools used in standardized medical 
communication. Since a working knowledge of disease is imperative for students in order to establish a foundation 
for building evidence based medication recommendation, building high-level cognitive skills that strengthen 
students’ knowledge base is a major goal of this course. 

In addition, as the PharmD program attracts very motivated students, their focus on high achievement has a 
positive impact on the instructional process as both students and instructors focus on continually evolving ability- 
driven learning outcomes recognized as valuable in the Pharmacy field. The downside of this environment is that 
students pass from generation to generation “recipes of success” and any major disruption in the instructional 
process has the potential to generate swift resistance from students. This resistance, in turn, can prevent faculty 
members from extending their attempts to change the instructional process to the point where they can fully evaluate 
the potential benefits of the proposed change. Therefore, a second critical element in the redesign of the instructional 
process in a PharmD program is a careful identification of those elements that can produce the maximum benefits on 
the learning process with a minimal disturbance. 

This paper will analyze the design and implementation of an active learning strategy to address the found 
gap in one module of the course that covered Injury, Inflammation, Repair and respectively Infectious and 
Immunological Diseases while minimizing the disruption in the instructional process and maximizing the benefits of 
active learning strategies. 


Instructional Intervention 


The Existent Instructional Process 

The focal course typically had 85 students and takes place in a traditional stadium style classroom with 
bolted-on chairs and tables. One important addition to the instructional process at the time of this study was the 
inclusion of iPads as required in-class note-reviewing and note-taking tools. Instructors were also stimulated to take 
advantage of the mobility features of this tool as they look for opportunities to enhance the learning process. While 
the instructional process in the focal course was comprised of traditional face-to-face didactic lecture, the instructor 
augmented it with focused questions using a virtual personal response system known also as virtual clickers. In 
addition, lectures often include case -based discussions followed by in-class clicker questions and application 
homework assignments. 
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Active Learning Instructional Strategies 


The development of technology stimulated a shift of attention from the structure and efficiency of content 
as knowledge delivery toward the effectiveness of the knowledge building through the engagement of learners in 
meaningful classroom tasks that mirror real-life learning situations (e.g. Brown, Collins, & Duguid, 1989; Collins, 
Brown, & Holum, 1991). In this context, active learning emerged as a pool of strategies focusing on how learner 
engagement can serve as a moderator for meaningful knowledge construction in classroom settings (Bonwell & 
Eison, 1991; Chi & Wylie, 2014; Pelley, 2014; Wolfe, 2006; Wolff, Wagner, Poznansky, Schiller, & Santen, 2015). 
While published research documented the benefits of active learning strategies especially for science, engineering 
and mathematics fields (e.g. Freeman et al., 2014; Gardner & Belland, 2012; Wenzel, 2014), findings also pointed 
toward potential barriers such as apparent student and faculty resistance (e.g. Scheyvens, Griffin, Jocoy, Liu, & 
Bradford, 2008; Walters, 2014; Wolfe, 2006) or the need for specialized infrastructure (Park & Choi, 2014). 

The rise of digital technologies that allow more seamless recording, storing and streaming of video files 
created opportunities for the flipped classroom, an active learning strategy that moves the focus from content 
coverage to learning by doing in classroom settings (e.g. Bergmann & Sams, 2012; Fladman, McKnight, McKnight, 
& Arfstrom, 2013). This strategy changed the nature of homework from application-driven tasks (e.g. problems, 
small projects) to content-driven tasks such as video lecture modules (e.g. Pierce & Fox, 2012). By shifting the 
content-driven direct instruction online, during the face-to-face time the instructor has the opportunity to engaging 
the whole class in case or problem-based activities, coach individual students and help them master that content or 
both. The major factors needed to ensure a successful flipped classroom are: (a) willingness to create flexible 
instructional environments; (b) acceptance of and engagement in a shift in learning culture; (c) intentional focus on 
the offering of the content that can maximize students’ learning both outside and as part of the classroom; and (d) 
continuous engagement of educators into own professional development through reflective practice (Fladman et al., 
2013). 

Because engagement and reflection through case or problem based strategies was traditionally part of their 
instructional strategies, adoption of flipped classroom in health professions (Bristol, 2014; Galway, Corbett, Takaro, 
Tairyan, & Frank, 2014; Gilboy, Fleinerichs, & Pazzaglia, 2015; Pierce & Fox, 2012; See & Corny, 2014; Simpson 
& Richards, 2015) was both an appealing and logical option. Flowever, a recent scoping review found that there is 
no formal conceptual framework used to design flipped classrooms (O'Flaherty & Phillips, 2015). In addition 
O'Flaherty and Phillips (2015) indicated that while research findings supported an increased engagement and 
acceptance of flipped classroom, there were still significantly large groups of students that expressed frustration due 
to perceived extra loading outside the regular classroom meetings. This finding will therefore support proposed 
approaches to combine flipped classroom and blended learning strategies to provide a distinctive learning space for 
online materials as complementary to the live classroom engagement (e.g. Baepler, Walker, & Driessen, 2014; 
Galway et al., 2014; Wolff et al., 2015). 

Following these findings from the literature, the proposed instructional process used a combination of 
blended learning and flipped classroom strategies to maximize the benefits of the two active learning strategies 
while minimizing their individual weaknesses. 

The Redesigned Instructional Process 

The decision was to redesign one three-week module that could ensure the highest potential to address the 
identified gap in students’ learning. Out of the six class meetings of this module, half became fully online activities 
while the other three were face-to-face meeting. The design of the two types of activities ensured that the 
instructional task were both complementing and supporting each other. For the online activities, the focus was on the 
content by providing students with 25-30 minute pre-recorded lecture modules but also included online case studies 
to provide a starting point for individual preparedness for live active learning meetings (see Figure 1). 

Of the three face-to-face meetings, one followed the traditional lecture with active engagement through 
clicker questions and the other two included a flipped classroom approach. The traditional lecture was set as the first 
one in the module and had the role of introducing new concepts that would not be feasible for an online, self-guided 
module. 


The two flipped-classroom sessions build on a combination of case-based learning and team-based 
learning. Each session started with a short review of the concepts covered in the online activity, followed by a quiz 
to test these concepts and finalized with the actual flipped classroom tasks. 
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Friday, October 10th Materials 

Enabled Statistics Tracking 

Attached Files O 2lnjury_2014.pdf (522 543 KB) 

□ 2lnjury_2014 (power point] pdf (2 05 MB) 

□ 2lnfectious Disease_2014 pdf ) (449 233 KB) 

□ 3lnfection_bactena_presentation_2014 (power point] pdf (1 915 MB) 

□ 2Sites of infection 2014 pdf 5 (248 68 KB) 

□ Groups SOAP note_Oct 13_2014 pdf Q (45 315 KB) 

Req uired Pre - le ct ure Ftea d'ng 

1 Introduction to Human Disease Pathophysiology for Health Professionals Chapters 4 and 26 

2 The Language of Medicine Chapters 14. 20. and 21 


Applic alien Activities 


1 Download or pnnt the Injury _2014. Infectious Disease_2014. and Sites of lnfection_2014 lecture notes 
A / 2 Listen to Injury Lecture and complete Case 1 

3 As a review of your microbtology/pathogenic microbiology pre-pharmacy classes access and view the 

4 Listen to the Infectious Disease Lectures and complete the sites of infection figure with the most comr 


Figure 1. Structure of instructional tasks for one of the two online modules (Blackboard screenshot) 


The major focus of the flipped classroom session was on creating SOAP notes (synthesis tools used in 
standardized medical communication) from a case study adapted by the instructor from actual clinical data. Because 
the classroom was a traditional stadium style classroom with bolted-on chairs and tables, the flipped classroom 
session used a modified Gallery Walk strategy. While the traditional Gallery Walk requires students to move across 
several complex problems or cases posted on the classroom walls on big Post-it® notes (e.g. Hogan & Cernusca, 
2011), in this course each group worked on one case on a single poster (Figure 2a). This strategy allowed the 
instructor to move from one group to the other and become an active facilitator of the learning process during this 
stage of the instructional process (Figure 2b). 



(a) (b) 

Figure 2. Flipped classroom session, (a) Modified Gallery Walk; (b) The instructor as active facilitator 


However, during the flipped session, multiple teams worked on the same case without knowing about this 
case overlapping. After each team created their own SOAP note, the instructor paired up the teams with the same 
case, asked them to bring their post-it note in the proximity of each other and encouraged them to discuss their 
outcomes. This reflection and negotiation step was a critical stage for internalization and enrichment of knowledge 
and skills. 

Finally, to complete the reflective part of this assignment and at the same time make the results of all cases 
available to the entire class for exam review (e.g. Hogan & Cernusca, 2011), each team then posted the SOAP notes 
in a dedicated Wiki page (Figure 3). 
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Figure 3. SOAP note posted as Wild for grading 

Because all students were required to have an iPad as part of this PharmD program, groups not only had 
access to all needed course and online public resources they needed to solve their SOAP note case, but also were 
able to start working on the Wiki posting during the flipped classroom session. The instructor gave each team a 
couple of days to review and complete the Wiki post before grading it. 

Research Questions & Methodology 

The exploratory questions for this study were: 

(1) Do students in the active learning group have higher scores that those in the lecture-based group on the 
exam SOAP application part directly associated with the active learning tasks? 

(2) Is there a transfer of knowledge to topics targeted by the active learning tasks, as reflected by students’ 
performance on specific exam items? 

The analysis and intervention periods occurred the fall of 20 13 for baseline measurements and fall 2014 
respectively for redesign measurements. Each class had 85 students. To test the homogeneity of the two groups, the 
instructor administered an in-class, unannounced, prior knowledge assessment during the first week of the course. 
The prior knowledge assessment included 1 8 multiple-choices questions created by the instructor to measure 
students’ expected knowledge for major topic such as respiratory, cardiovascular, renal, gastrointestinal and 
oncology, considered important for a good performance in the pathophysiology course. 

The content of the course remained the same between years. However, for this intervention instructional tasks 
were adapted using active learning strategies for the fall of 2014 class. To evaluate the impact of the active learning 
at the exit level, we used assessment items from the final module examination that included case based multiple- 
choices questions and a SOAP note. The instructor kept all examination multiple-choices items associated with the 
redesigned module either identical of very similar and the SOAP note used equivalent cases to analyze. To test the 
impact of active learning strategies on knowledge retention, we identified a total number of 16 multiple-choices 
questions that targeted SOAP notes elements such as infectious diseases and immunologic disorders. Of these three 
topics, the immunologic disorders was a new one, introduced in the module we analyzed and included in several 
case studies used with SOAP notes. 
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Finally, to gauge students’ perception of the active learning module, students from fall 2014 cohort completed a 
short online exit survey administered on iPads in classroom using Qualtrics®. This study received the IRB approval 
to guarantee the needed protection of the students that volunteered to participate. 


Results and Interpretation 


An independent-sample t-Test indicated no statistically significant differences between the prior knowledge 
of the two groups, suggesting that the two cohorts were homogeneous at the entry point. 

For the performance analysis exam results indicated statistically significant increases for the active learning 
group for SOAP notes administered in the module examination, t( 1 3 1 ) = -6.94, p < .001. Figure 4 synthesizes these 
mean SOAP notes scores for the two groups. 


89.41 


75.12 


This result confirms the expectation 
that active learning strategies build 
around this tool would increase 
students’ performance on near-transfer 
tasks. 

The analysis of the 16 
multiple-choices items associated with 
the transfer of knowledge for the topics 
covered during the active learning 
tasks, indicated a statistically 
significant increase of mean scores, 
t( 1 6 1) = -2.11, p < .05. In addition, the 
subgroup of multiple-choices items 
that assessed students’ knowledge of 
immunological diseases showed also a 
statistically significant increase in 

I mean scores, t( 143) - 6.94, p < .0001, 

Figure 4. Mean values for SOAP notes in the module examination with an even bigger increase in 

students’ gains for the group that was 
exposed to active learning (Figure 5). 

Since, as we priory mentioned, immunological disease was a new topic introduced in the module that was the focus 
of this study this increased gain strengthens the potential positive impact of active learning when new topics are 
introduced in a course. 
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These findings show a 
positive support for both exploratory 
research questions in this study. The 
beneficial impact of active learning 
was reflected not only on the exam 
tasks directly linked to the SOAP 
notes but also in the strengthening of 
the knowledge base associated with 
these SOAP notes. 

Students’ answers to the 
open-ended questions in the exit 
survey indicated that most of the 
students liked the active learning 
and the hands-on activities 
associated with them. 


■ FS14 m FS13 

Figure 5. Mean values for multiple-choices (MC) questions associated with 
the SOAP notes in-class active learning tasks 
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For example, when asked what helped their learning during the active learning sessions, students 
mentioned “specific examples of how concepts are applied”, or that they . . really enjoyed all of the hands-on 
learning”. Some of the students provided some more detailed accounts of their perception of the online course: 

1 found that her teaching style which includes class involvement highly beneficial. Personally, I enjoy 
learning new material. The more challenging the material, the more I push myself to learn it. I have found 
that in other classes where the material is more of a review I tend to slack off not pay as much attention as I 
should be. 1 have enjoyed her portion of this class, and I feel that I have learned a great deal of 
information. 

DR. [instructor name] HAS BLENDED HER LECTURES BOTH TRADITIONAL IN-CLASS LECTURE 
AND ON LINE.THIS GAVE ME THE CHALLENGE AND WHAT IS THE BEST WAY FOR ME TO 
LEARN. HER ON-LINE LECTURE WAAS VERY HELPFUL TO ME IN THAT I WAS ABLE TO GO 
BACK AND LISTEN TO WHENEVER I FELT LIKE I NEEDED 

However, some of the students did not like the online lectures or the Wiki task following the class active 
learning sessions. Most of students’ complains on the online lectures were linked to the time required to listen to 
them and their lack of engagement compared to the face-to-face lectures. As for the Wiki tasks, most of students’ 
complains were related to the technical difficulties (e.g. inability to have multiple contributors at the same time in 
Blackboard’s Wiki tool) and lack of group dynamics when compared to the face-to-face group meetings. This type 
of negative perceptions is in line with previously reported findings on the implementation of active learning 
activities (Baepler et ah, 2014). 


Conclusions and Further Research 

The relative big size of the class and the physical limitations of the classroom posed some challenges in the 
implementation of the active learning strategies. However, a careful design and an effective implementation of 
active learning strategies helped produce positive results on students’ performance without major disruptions in the 
instructional process. 

In addition, the benefits of active learning strategies have the potential to not only improve the cognitive 
skills directly associated with the active instructional tasks but also to strengthen the knowledge base engaged in this 
process. The instructor plans to extend this strategy to other modules in the course, and provide more support for the 
individual and group online activities. 
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Abstract 

This completed study examined the effects of pause time and types in instructional animations. A total 
of 170 college students were randomly assigned to one of the five different pause types (plain pause, passive 
reflection, active reflection, passive prediction, and active reflection) under either a long or short pause time 
condition. The results showed that all groups with reflection and prediction activities outperfonned the plain 
pause group in the multiple-choice recall test. Essay transfer test scores were greater with short pause time than 
long pause time, and were positively associated with word counts typed in the test. This study provides useful 
information to effectively apply the segmenting principle to an instructional animation. 

Introduction 

Animations can be useful instructional units to represent the procedure of complicated concepts and 
dynamic motions (Lin & Atkinson, 2011). However, they are not always superior to the matching static images 
in that the fleeting infonnation of the animation may require extra cognitive efforts to the learners (Mayer, 
Hegarty, Mayer & Campbell, 2005; Spanjers, Wouters, van Gog & van Merrienboer, 2011). In other words, 
individuals have limited capacity of working memory in holding multiple bits of infonnation and successfully 
remember all the infonnation (Baddeley, 2007). When the load additionally required for processing the 
animation exceeds the learners’ cognitive capacity, cognitive overload occurs (Baddeley, 2007). Therefore, the 
animation for instructional purposes should be strategically designed. 

The segmenting principle has suggested that cognitive load imposed to process the instructional 
animation can be reduced when the animation is presented in segmented pieces (Mayer, 2009; Spanjers et al., 
2011). Specifically, pause can gain extra time between segments of the animation, so learners may save time to 
process information in the previously presented segment (e.g., Hasler, Kersten, & Sweller, 2007; Mayer, Dow, 
& Mayer, 2003). More importantly, a recent study (Cheon, Chung, Crooks, Song, & Kim, 2014) demonstrated 
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that when pauses contain meaningful cognitive activity (i.e., reflection guide or free recall test), learning can 
improve without imposing additional mental efforts for the instruction and tests. Prediction activity (e.g., asking 
to predicting what they will learn in the next segment) can play a meaningful role during the pauses. Specifically, 
a prediction prompt likely helps learners prepare to learn and tie with their prior knowledge (Hegarty, Kriz, & 
Cate, 2003; Lim, Buendia, Kim, Cordero, & Kasmer, 2010). However, there has been little research exploring 
the relative benefits of different pause types in instructional animations. 

In addition, the effects of pause time on learning performance are unclear. For example, longer time 
during pauses may interfere with bonding the relationships between segments. Whereas, shorter pause time may 
not provide a sufficient amount of the extra time for the learners to completely think what they learned or will 
learn. Thus, the current study aims to investigate how pause time and types influence learning performance (i.e., 
multiple-choice recall and transfer tests and essay transfer test) in an instructional animation. Further, while 
employing essay transfer test in this study, we also aim to examine the relationships between the number of 
words counted in the test and learning performance. Therefore, specific research questions are presented as 
follows: 

• How do pause time and type in an instructional animation influence learning performance (i.e., 
multiple-choice recall, multiple-choice transfer, and essay transfer tests)? 

• How do word counts used in the essay test correlate with the recall test scores, transfer test scores, 
and essay test scores? 

Method 

Participants 

We collected data from 170 undergraduate students at a large southwestern university (female: 85, 
male: 85; freshman: 7, sophomore: 31, junior: 60, senior: 72). 

Instructional Animation 

This study used a 694 second-long instructional animation about the science of persuasion, which 
consists of system-controlled, an introduction part (76 seconds) and six shortcuts (i.e., principles) with 
animation and narration (see Figure 1) — (a) Reciprocity (106 seconds), (b) Scarcity (64 seconds), (c) Authority 
(113 seconds), (d) Consistency (96 seconds), (e) Liking (85 seconds), and (f) Consensus (154 seconds). So, the 
entire instructional animation was segmented into seven chunks by the introduction part and six principles. 



Figure 1. A Screenshot of Instructional Animation 


Data Collection 

This study used a 2 (pause time: long, short) x 5 (pause type: plain, passive reflection, active reflection, 
passive prediction, active prediction) between-subjects design. All the participants spent the same amount on all 
the seven chunks of the animations. The long pause groups had the same amount of pause time as provided for 
each segment’s length; thus they spent 1,3 12 seconds. Whereas, the short pause groups spent half time of the 
segments’ length during the pauses. The plain pause group was given a message “You will be moving the next 
animation” during all the pauses. Reflection and prediction groups were asked to reflect the summary of what 
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the principle means after the segmented animation or to predict the summary of the principle before the 
segmented animation. Active groups were asked to write the summary directly to the screen, whereas passive 
groups were asked to just think it in mind. 

Learning performance was measured using 12 multiple-choice questions for recall test, 6 multiple- 
choice questions for transfer test, and 6 questions for essay transfer test. Word counts were counted from the 
participants’ responses to the essay transfer tests. Recall test scores were summed and other test scores were 
averaged for data analyses. 


Results 

Prior knowledge was not significantly different for pause time (F( 1 , 160) = 3.18 ,/? = .08), pause type 
(F(4, 1 60) = .530, p = .7 1), and the interaction between time and type (F( 4, 1 60) = 1 .29, p = 216). 

Regarding multiple -choice recall test scores, pause type had a significant main effect (F( 4, 160) = 3.75, 
p = .006). No significant effect was found for pause time (F(l, 160) = .12,/? = .725) and interaction between 
time and type (F( 4, 160) = 1.21 ,p = .311). The post-hoc test indicated that plain pause was lower than other 
types (p = .008). 

Regarding multiple -choice transfer test scores, there was no significant effect of pause time (F( 1 , 160) 

= .39 ,p = .536), pause type (F( 4, 160) = .547, p = .701), and the interaction between time and type (F( 4, 160) = 
1.17,/? = .328). 

Regarding essay test sores, pause time had a significant main effect (F(l, 160) = 6.98,/? = .009). No 
significant effect was found for pause type (F( 4, 160) = 1.53,/? =.196) and the interaction between time and type 
(F(4, 160)= 1.90,/?= .112). T-test analysis found that shorter pause time was more effective than longer pause 
time (7(168) = -2.46,/? = .015). 

Last, correlation analysis indicated that word counts were significantly correlated with essay transfer 
test scores (/-( 1 68) = .25,/? = .001) but not correlated with recall test scores (?'(168) = .15,/? = .051) and transfer 
test scores (/-(168) = .04,/? = .607). Table 1 describes means and standard deviations of all test scores. 

Table 1. Means and Standard Deviations for All Tests 




Long Pause Time 





Condition 





(n = 

101) 




Plain 

Passive 

Active 

Passive 

Active 


Pause 

Reflection 

Reflection 

Prediction 

Prediction 


(n — 17) 

(n = 23) 

(n = 23) 

(??= 19) 

(?? = 19) 

Multiple-Choice 

14.00 

16.52 

16.57 

15.26 

15.68 

Recall Test 

(3.50) 

(1.81) 

(15.26) 

(2.33) 

(1.92) 

Multiple -Choice 

3.94 

4.57 

4.35 

4.00 

4.00 

Transfer Test 

(.90) 

(.79) 

(1.11) 

(1.49) 

(1.16) 

Essay Transfer 

1.86 

2.43 

2.33 

2.17 

1.84 

Test 

(.57) 

(.54) 

(.66) 

(.95) 

(.71) 



Short Pause Time 





Condition 





(« = 

= 69) 




Plain 

Passive 

Active 

Passive 

Active 


Pause 

Reflection 

Reflection 

Prediction 

Prediction 


(»= 13) 

(« = 15) 

(n = 23) 

(?? = 14) 

(?? = 14) 

Multiple-Choice 

14.38 

16.07 

15.46 

16.50 

16.29 

Recall Test 

(3.62) 

(2.25) 

(3.69) 

(1.45) 

(1.07) 

Multiple-Choice 

4.08 

4.27 

4.00 

4.57 

4.50 

Transfer Test 

(1.32) 

(1.16) 

(1.23) 

(1.22) 

(1.16) 

Essay Transfer 

2.32 

2.60 

2.12 

2.55 

2.60 

Test 

(.74) 

(.80) 

(.96) 

(.93) 

(.69) 


* Standard deviations are presented in parenthesis 


Discussion 

The results of this study showed that the plain pause which did not require meaningful cognitive 
activity reported the lowest test scores in most learning tests, even though the differences were not significant. 
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Thus, the findings suggest the benefits of placing a meaningful cognitive activity during pauses between 
segments of an animation, when the segmenting principle is applied. 

Moreover, although we did not find significant differences between reflection and prediction activities, 
because of the different roles and expectations, we can conjecture that reflection helped learners re-organize 
what they learned and link all the six principles (Moreno & Mayer, 2007). On the other hand, prediction could 
function to facilitate the learners guess incoming information and prepare what they would learn, and further 
integrate all the principles (Goldberg, Casenhiser, & Sethuraman, 2005). 

Furthennore, our findings showed pause time influenced higher-order learning outcomes, essay transfer 
test scores, regardless of pause types. We conjecture that too long pause time may impose unnecessary load, 
such as extraneous load (e.g., Cheon et al., 2014), thus interrupting learners’ attention and engagement with the 
instructional animations. 

Additionally, the number of words typed in the essay transfer test were significantly associated with the 
test scores. Thus, when the segmenting principle is applied, active essay (i.e., typing more words) likely cause 
more meaningful and accurate thinking process. 

This study revealed several limitations. The instructional animation we used has a specific topic in 
psychology, the science of persuasion. Its content introduces each of six different principles on how to persuade 
people. However, because it was not procedural knowledge, it could be hard for learners to properly predict the 
next phase based on the previous segment. Thus, future study needs to select a different learning topic, 
containing sequential steps, which may anticipate the potential effects of prediction activity. Second, the tenns 
‘passive’ and ‘active’ need to be clearly defined. For ‘active’ conditions, previous studies provided specific 
learning tasks, such as embedded questions or free recall tests. However, our study simply asked learners to 
write the summary what they watched, which was not much different from ‘passive’ activity asking thinking 
about what they learned. Last, as our findings showed pause time influenced essay transfer test scores, future 
study may need to have multiple pause time conditions. 
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Research Purpose & Approach 

The purpose of our research study was to understand features of CBL eLearning designs that, when 
combined, lead to ability to address ethical dilemmas in genomics and also lead to student satisfaction with learning. 
An interdisciplinary team of experts in Genomics, Philosophy, and Instructional Design was formed to design, 
develop, implement, and evaluate an online three credit-hour Ethics in Genomics course. The course goals were to 
engage students in thought-provoking conversations about ethical and societal questions raised by actual and 
potential scientific advancements, introduce students to connected disciplines outside their area of expertise, 
facilitate student research opportunities that address ethical and social questions raised by genomics technologies, 
enable student engagement with leading research scholars, develop ingenuity in students, and produce graduates 
who are cognizant of far-reaching ethical issues and consequences associated with science and technology. The 
project goal was to collaboratively develop a university level, case-based, online course composed of electronic 
learning modules on Ethics in Genomics for broad dissemination through a MOOC. 

The team designed the Genomics Ethics course iteratively over three years of funding by NSF and in three 
phases: 1) face-to-face, 2) blended, and 3) fully online. A design model based on ADDIE was applied for course 
development. The research approach that guided our study is product design and development (Richey & Klein, 
2007). Our project emphasis is grounded in comprehensive design and development, guided by case study research 
of three genomics ethics course deliveries. 
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Problem 


Developing high quality instruction involves complex transactions among designers and instructor/experts. 
College professors typically do not know how to design online courses. They most commonly are guided by an 
instructional designer to develop direct instruction, using the following format: Units consisting of attention getters, 
objectives, content, assignments, & assessments. Often, within such design teams, courses are reduced to a series of 
captured lectures with supportive readings. Clarification is needed for how to guide professors to develop authentic 
e-Learning, that is, online instruction that prompts students to perform real-world tasks in the field of study 
(Herrington, Reeves, & Oliver, 2010). 


Proposed Solution 

As a solution for developing high quality online instruction, and to facilitate the necessary complex 
transactions between instructional designers and instructor experts, our team adopted Aleckson & Ralston-Berg’s 
(2011) Micro-collaboration Model (Cifuentes, Park, McQueen, & Riggs; submitted). The five factors detailed in the 
model are power relationships, management, formative evaluation, shared culture, and momentum. Additionally, 
creating authentic experiences for students were ensured through application of the CBL, PBL, and PjBL Model for 
ID of Authentic, Experiential Learning developed by Cifuentes, Mercer, Alvarez, and Bettati (2010) was used to 
guide the construction of meaningful, engaging, course content (see Figure 1). 

The researchers applied design and development research methods with each delivery phase to determine a) 
the impact of online case-based instruction on students’ abilities to address ethical dilemmas in genomics, b) 
students’ satisfaction with course features in each of the three iterations, and c) what revisions were needed 
following the first two course iterations and deliveries to improve instruction. 

Methods 

The research study within the design and development project involved distance-based implementation of 
the course in two public universities over 200 miles apart within the same system: Texas A&M University, where 
the Norman Borlaug Institute for International Agriculture designs and implements science-based development 
projects and training programs that guide the phases of agricultural industry from production to consumption, to 
fight hunger and poverty among small holder agricultural communities of the developing world, and Texas A&M 
University-Corpus Christi. The distance between the institutions brought relevance to development processes for 
online instruction designed to reach distantly separated students. 
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Case, Problem, and Project- Based Learning Model 
for Instructional Design of 
Authentic and Experiential 
Learning 




B M—WI 




Figure 1. Case, problem, and project-based learning model that guided course design. 


The research team consisted of four instructor-experts (IE), three instructional designers (ID), and one 
graduate research assistant. One of the IEs was a professor of Genomics who conducts genomics research and 
teaches genomics courses. Another IE was a professor of Philosophy at the same university and an expert in applied 
ethics who conducts research on issues and problems relating to animal and environmental ethics. The third IE was a 
doctoral student in Philosophy studying animal ethics, and the fourth IE was also a doctoral student in Philosophy. 
Each IE had taught or helped to teach a face-to-face course on genomics ethics before, but none of them had taught a 
blended or fully online course prior to designing, developing, and teaching this course. One of the IDers was a 
professor and director of distance education at a public university in the United States with nearly 30 years of 
experience as an IDer. Another IDer was a postdoctoral researcher who held a doctoral degree in instructional 
design. A third instructional designer was a curriculum coordinator for an engineering extension agency. Her role in 
the project was to develop video content and coordinate open access to the course for broad dissemination. In 
addition, a doctoral level research assistant helped during stages of the research process. 


Research Paradigm 

This study can be described as “product research” as it explores the entire design and development process 
from analyses to evaluation. Richey and Klein (2008) define design and development research as “the systematic 
study of design, development, and evaluation processes with the aim of establishing an empirical basis for the 
creation of instructional . . . products and tools and new or enhanced models that govern their development” (p. 748). 
We continuously conducted formative evaluation focusing on course structure, unit/module development and 
content, and development of the eLearning components as they were implemented with students. Therefore, student 
outcomes and levels of satisfaction guided revision. 


Participants 

The study participants for the first course delivery were graduate and undergraduate students enrolled in a 
philosophy course, Genomics and Society, during the Spring 2014 semester. Eighteen students were enrolled in the 
class and all of them consented to participate. Thirteen were graduate students and five were undergraduate students 
with one sophomore, two juniors, and two seniors. They represented a range of majors including agriculture. 
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genetics, geosciences, and philosophy. There were ten females and eight males. Students reported their ethnicity as 
thirteen Caucasians, two Hispanics, one Asian, one African American, and one other. 

In the second course delivery, participants were undergraduate and graduate students enrolled in 
Philosophy 489/689 Genomics and Society during the Spring 2015 semester. The course was delivered to seventeen 
students with five consenting to participate. There were five females and no males, with a demographic of three 
white and two Latina students. The decline in enrollment can be attributed to the class’s change in status as one that 
did not fulfill a requirement of the core curriculum. We attribute the decline in study participation to the effects of 
face-to— face vs. online recruitment. Other researchers have found that personal presence when gaining consent and 
collecting data leads to a higher response rate (Granello & Wheaton, 2004). 

Data Sources 

Data sources for our study included scores on case analyses, scores on discussions, student focus-groups, 
student satisfaction surveys, background genomics opinion surveys, instructor and course appraisal evaluations, unit 
evaluations, course design features themselves, and design and development documents. 

Data Collection 

We collected data continuously across three years. We occasionally experienced issues with disagreements 
between content-experts and instructional designers regarding course content, course design, and data generation For 
instance, IDers wanted each unit to include a case for students to address, while instructors wanted students to 
choose two of the eight cases to address; and similarly IDers wanted a discussion forum in each unit, while 
instructors wanted students to select discussion forums in which to participate. As a general rule, IDers deferred to 
instructor/experts when making final design decisions. Data collection processes across both iterations began by one 
of the IDers calling for student participation through administration of consent forms. Students either signed a letter 
of consent or non-consent to participate or declined participation in the study. 

The scores on case analyses for the two iterations were derived by the instructors using a case-analysis 
rubric developed and revised for each course iteration. They were obtained during normal course processes and were 
exported as reports from the e-campus grade center. Discussions scores across the iterations were derived by the 
instructors who filled out Discussion Forum rubrics. The scores were collected through the online grade center in the 
e-campus learning environment. At the conclusion of the first two iterations, students gathered face to face for a 
one -hour long focus-group, to share reactions to their course experiences. The focus-group protocols included 
introductions, an explanation of design and development goals and processes, and an explanation that the focus- 
group would be audiotaped or compiled. The general question posed to students was, “What do you think of the 
course?” Probing questions followed that were designed to elicit suggestions for adding design features to improve 
the course and determine satisfaction levels of students. The focus-groups were audio-recorded and transcribed 
verbatim. Survey data was collected continuously through two course iterations and end-of-course evaluations were 
administered according to university policy. 


Data Analyses 

The quantitative data, scores on case analyses and scores on discussions, were analyzed by constructing a 
summary of means table to represent student scores on cases as percentages and scores on discussions as comparable 
means. A content analysis was used to code, categorize, and theme the qualitative data from student focus-groups, 
background genomics opinion surveys, instructor and course appraisal evaluations, unit evaluations, course design 
features, and design and development documents. Student responses were used to inform design and development 
changes across iterations. The research team gained consensus on the codes that emerged from the data (Fereday & 
Muir-Cochrane, 2006). 


Results 

The course design elements were constructed with the understanding that the course would ultimately be 
delivered fully online. The structure of the course consisted of eight units: 1) Introductory Materials: Genetics, 
Ethics & Policy, which served as an introduction to case analysis; 2) Genomics and Synthetic 
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Biology/Microorganisms/Biofuels; 3) Genomics and Crops; 4) Genomics and Domesticated Animals; 5) Genomics 
and Wildlife; 6) Genetic Therapy and Human Enhancement; 7) Genetic Data and Privacy; and 8) Final Case. 

The first research question regarding students’ abilities to address ethical dilemmas in genomics, was 
derived from graded case analyses and online discussions that were part of normal course processes across all three 
iterations. The trend for both graduate and undergraduate students was that their abilities to analyze cases improved 
with practice over the term. For instance, from the beginning to the end of semester graduate students scores 
improved from 91% to 93% in the first iteration and from 91% to 95% in the second iteration. Undergraduate 
students improved even more over time, scoring a mean of 80% and 83% in the first case and 92% and 93% by the 
end of the term. Scores did not improve from one iteration to the next indicating that design features of the case 
analyses assignment did not improve (see Table 1). 

Table 1. 

Student grades on case analyses 


Administration 

1 st Case Analysis 

2nd Case Analysis 

3 rd Case Analysis 

1 st Iteration to Grad 

Students 

91% 

94% 

93% 

1 st Iteration to 

80% 

89% 

92% 

Undergraduate Students 

2 nd Iteration to Grad 

91% 

68% 

95% 

Students 




2 nd Iteration to 

Undergraduate Students 

83% 

75% 

93% 


During the first iteration when only two discussions were required of students, scores improved across the 
term. On the other hand, scores on discussions generally declined slightly in the second iteration when both graduate 
and undergraduate students participated in six discussion forums. Scores did improve from one iteration to the next 
indicating that design features of the discussions improved (see Table 2). 


Table 2. 

Means of student scores on discussions (range = 1-5) 


Administration 

1 st Iteration to 
Grad Students 

1 st Iteration to 
Undergraduate 
Students 
2 nd Iteration to 
Grad Students 

2 nd Iteration to 
Undergraduate 
Students 


pt 

2nd 

3rd 

M 

M 

M 

4.35 

4.60 

— 

4.23 

4.76 




4.75 

4.70 

4.90 

4.60 

3.90 

3.80 


4th 

5 th 

6th 

M 

M 

M 





4.75 

4.20 

4.50 

4.50 

4.40 

4.40 
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Qualitative Findings 


Design and development documentation revealed that instructors graded according to expectations that 
students would get better at case analyses and discussion. As a result, they graded more leniently in early cases and 
discussions. Over time, instructors claimed that they observed increasing sophistication in students’ abilities to 
critically evaluate ethical dilemmas. Instructor feedback on a student’s second case study during the first iteration of 
the course was positive, “this is a really interesting and thoughtful paper. You raise a wide range of issues, consider 
them in depth, develop them carefully and draw on appropriate literature effectively. This shows real understanding 
of the issues at stake”. In the third iteration of the course, students showed improvement on their ability to address 
ethical dilemmas through discussion: “As usual, you make both thoughtful and useful contributions to the 
discussion, drawing appropriately on the readings, and adding both to the more practical aspects of the discussion 
(eg. in your first post on reducing environmental risk) and to the more theoretical aspects of the discussion in your 
other posts.” Instructor feedback on the final case analysis of the third iteration also demonstrated student 
improvement, “Overall, 1 think you did a really nice job here, student - this is the best thing of yours I've read. It's 
thoughtful and discusses the issues carefully and in an interesting way.” 

Students’ Changes 

Survey results demonstrated a change in student beliefs toward ethics and genomics topics during the first 
class iteration was an interesting observation encountered. We found that as the course progressed, students 
increasingly believed that GMO generated dings are safe, that it is OK to do anything scientists like to animals if 
humans benefit, that it is less important to protect animals from extinction, and that GMOs do not pose a threat to 
the environment. This shift in beliefs indicated a possible deviation from our goal of encouraging students to think 
critically by using ethics to develop their own arguments towards dilemmas in genomics. In fact, it may reflect the 
perspective of the Borlaug Institute. 

Student Satisfaction and Suggested Revisions 

The second research question of student satisfaction with design features was informed by data from 
student focus-groups and student satisfaction surveys. In the focus-group conducted at the end of the first iteration 
students expressed that they particularly enjoyed the inclusion of expert lecturers as part of the course: “I thought the 
guest speakers were fantastic. It was a wide variety.” Alternately, students shared their somewhat negative views 
toward the case analyses assignments; they felt limited by the specific directions to write four pages for each case 
analysis. Furthermore, students shared that they had difficulty engaging in the discussions, they felt that tiying to get 
into conversation in the large discussion forums was too much. They suggested that the researchers break the 
discussions into smaller groups, common practice in online courses, which the IDers did for the second iteration. 

During the focus-group held at the end of the second iteration, students gave instructional designers useful 
suggestions for further course revision. A revision suggestion offered by students was to break each unit into smaller 
subunits; they felt overwhelmed by “too much content to scroll through”. Second, students voiced that they, “would 
like a choice [among cases].” Students had positive views of discussion revisions implemented by the researchers; 
they liked the redesign of the discussion that changed from asking big questions that tended to elicit essays to small 
topics that elicited interchange. 

Student satisfaction surveys employed during the third iteration allowed researchers to gain insight into 
student satisfaction with course design features. Students enjoyed the pacing of the course content, “I liked that we 
got to spend 2 weeks on each unit. . ..I think that gave us enough time to do the reading assignments and really get a 
good understanding of what was going on.” Additionally, students gave positive feedback regarding course 
organization stating, “The topics were well organized and logically followed each other.” Finally, students were 
appreciative that instructors worked to maintain a personal presence, even though the course was offered completely 
online, “Attempting to schedule online meetings [web-conferences] each unit was a good idea.” 

Students provided feedback that revisions made to improve course outcomes and student satisfaction 
following each of the first two course iterations and deliveries and during the delivery of the third iteration paid off. 
For instance, revision of course unit structure was a major revision informed by student input and collaboration 
between instructor experts and instructional designers. The course unit structure of iteration one that informed future 
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revision was as follows: 1) Scenario; 2) Background; 3) Discussion; 4) Video and readings; and 5) Case Study. 
Following the initial iteration, instructional designers conversed with instructor experts, who agreed to reorganize 
the unit structure for easier navigation and understanding. The unit structure of the most recent iteration was as 
follows: 1) Essential questions; 2) content including videos; 3) Selected issues in depth; 4) readings; 5) Discussion; 
and 5) Case Study. 

Across the three iterations, students provided the instructional designers with valuable input about the case 
studies they experienced during their time in the course. They gave the following commentary during focus-group 
interviews. “Some initial lectures on the basics of moral theory would be helpful”, students felt they did not always 
have the necessary ethics background to understand course content. Students were confused by some of the 
terminology used in the case studies, describing that “there were some vocabulary problems.” An additional opinion 
was that it was difficult to personally relate to the scenarios within the cases, “make the cases more relevant to our 
experience”. Another student elaborated, “We’re supposed to be writing as someone from the FDA, and that was 
super awkward.” Many of the students also felt that the case studies were too linear, “the prompt [in the cases] was 
leading you to take a stance. . ..I thought the stance was taken for us.” Finally, students related that having, “a good 
example of what you’d actually like [the case analysis] to look like”, would be help them to write their case 
analyses. 

Essential Questions 

During the focus-groups, the instructional designers asked students how they felt about the addition of an 
essential questions feature, a set of guiding questions for students to consider while they studied each course unit. 
The student responses were positive, suggesting that the feature would be helpful to their learning, one student 
explained, “I think that’s kind of what we need.” 

Discussion Forums and Rubrics 

The instructional designers asked the student focus-group participants how they felt about the discussion 
forums, many students were concerned about the online discussion environment: “There have to be a lot of those 
forums. . .maybe one for each unit.” Students’ views on the course discussions were not completely positive. In our 
case, stringent requirements of rubrics yielded generic, essay like responses. Many complained of feeling 
constrained to give a specific answer in an essay type format. The students also described that they felt their 
responses would not be original, a repeat of someone else’s, especially if they were one of the latter postings. In 
review and reading of the student discussions during data analysis, multiple discussion answers looked about the 
same, validating the concerns of the students. 

Guest Speaker Videos 

The data collected from the student focus-groups revealed that students enjoyed the design feature of the 
video recorded expert guest speakers integrated into each unit of the last two iterations of the course. One student 
described the quality of the guest speakers, “They were very knowledgeable, I was impressed with their celebrity.” 
However, the video was captured with one camera and microphone in a noisy lecture hall where the guest speakers 
gave live lectures accompanied by PPT presentations. Therefore, the quality of the captured video was low, making 
it difficult to focus for a full hour in the online, archived lecture. 

Guidelines 

Based upon our design and development experience, we recommend the following features of our CBL 
eLearning design that, when combined, led to ability to students’ abilities to address ethical dilemmas in genomics 
and also led to their satisfaction with learning are recommended below: 

• Include essential questions at the beginning of each unit to prompt critical thinking. 

• Provide students with a foundational content from the start rather than scattered across the course. 

• Include a glossary to promote understanding of course content. 

• Provide examples of excellent case analyses as well as explanations of common errors; this 

allows clarification of expectations to students. 
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• Cases should be relevant and relate to students’ experiences. Do not require students to play roles 
of someone they are not and will never be. 

• Provide more than one case in each unit so that students have choice. 

• Create ambiguous cases where there is no single, academic point of view. 

• For efficiency of discussion manageability, organize discussions into groups of no more than 
eight students. 

• Include opportunities for discussion in each unit. 

• Create a rubric that addresses substance, collaboration, and contribution to dialog rather than 
quantity of entry to evaluate students’ participation in discussions. 

• Invite expert guest speakers and when recording their presentations, arrange for high quality 
video-capture. Most teleconferencing systems allow for capture, archiving, and linking. 

Conclusions 

Situated learning theorists propose that the most important undertaking when designing authentic learning 
environments is generating ill-defined, complex tasks that engage the learner over extended periods of time and lead 
to decision making, analysis, reflection, synthesis, and product development. What enables learners to successfully 
complete a whole task is clear expectations and substantial learning resources and supports for the component skills 
or part-tasks that are prerequisites for completion of the whole-task (Merrill, 2013). Additionally, Catrambone 
(1994) showed that when students have explicitly worked through sub-tasks to address a problem, they are better 
able to apply those subtasks to a novel problem-solution. To provide for situated cognition, cases must be written 
with students’ interests in mind. They must take on roles to which they can relate. Asking them to play roles as 
farmers in Mali or Monsanto engineers was not helpful for our students who look forward to real careers in the field 
based upon their life experiences. Cases should be designed to reflect roles that they might one day fill. 

One of the limitations of the study was the low number of student participants in the third iteration of the 
course, out of eighteen students, only five were able to be used as data sources. A factor related to lower 
participation numbers was course mortality. Additionally, the completely online environment gave the researchers’ 
less access to participants, making their participation less likely, this may have led to the lack of participation in the 
third iteration of the course. 

Implications for Practice and Need for Future Studies 

Institutions of higher learning can benefit from the use of online courses in teaching ethics related content 
and evaluating students’ ability to address ethical dilemmas through the use of case studies and discussions. Rubrics 
are a valuable resource for students in online courses as they clearly communicate what is expected in writings such 
as case analyses and discussion. Instructors can use rubrics to clearly communicate what they expect from students 
taking their course and to efficiently grade assignments. In our research, we found that there is a need for faculty 
members and instructional designers to know more about attributes of successful eLearning designs. Futures studies 
that further explore the specific features of authentic e-learning, and their success in educational practice are a 
welcome contribution to the body of literature surrounding instructional design and online learning. 


65 



References 


Aleckson, J. D., & Ralston-Berg, P. (2011). Mindmeld: Micro-collaboration between eLearning designers and 
instructor experts. Madison, WI: Atwood Publishing. 

Catrambone, R. (1996). Generalizing solution procedures learned from examples. Journal of Experimental 
Psychology’: Learning, Memory, and Cognition, 22(4), 1020-1031. 

Cifuentes, L., Park, S. W., McQueen, J., & Riggs, P. (submitted). Collaboratively developing e-learning modules 
and courses across a distance. Educational Technology’ Research and Development. 

Cifuentes, L., Mercer, R., Alvarez, O., & Bettati, R. (2010). An architecture for case-based learning. TechTrends, 
54(6), 44-50. 

Fereday, J., & Muir-Cochrane, E. (2006). Demonstrating rigor using thematic analysis: A hybrid approach of 

inductive and deductive coding and theme development. International Journal of Qualitative Methods, 5(1), 
80-92. 

Granello, D. FI., & Wheaton, J. E. (2004). Online Data Collection: Strategies for Research. Journal of Counseling & 
Development, 82(4), 387-393. 

Flerrington, J., Reeves, T. C., & Oliver, R. (2010). A guide to authentic e-learning. New York: Routledge. 

Merrill, M. D. (2013). First principles of instruction. San Francisco, CA: John Wiley & Sons. 

Richey, R. C., & Klein, J. D. (2007). Design and development research. Mahweh, NJ: Lawrence Erlbaum 
Associates. 

Richey, R. C., & Klein, J. D. (2008). Research on design and development. In J. M. Spector, M.D. Merrill, J. van 
Merrienboer, & M. P. Driscoll (Eds.), Flandbook of research for educational communications and 
technology (3rd ed., pp. 748-757). Mahwah, NJ: Lawrence Erlbaum Associates. 


66 



A Comparison Study of a Face-to-Face and Online Writing Courses 


Ryan Eller, Bude Su, Karen Wisdom 

California State University Monterey Bay 

Abstract 

Online education is an ever-growing aspect of many students’ higher education learning experience. Its 
proliferation into each and every academic discipline is becoming an ever-present entity for university-level 
academic writing instruction and coursework. Through collecting pre-survey, mid-semester interview, and post- 
survey data in online and face-to-face sections of a same undergraduate writing course at a public west coast 
university, this research aims to promote a further understanding of student perceived readiness, student attitudes, 
student perceived ability, and their effects on student success in both online and face-to-face courses. Many students 
have preferences of how online instruction should be designed. While not every student share similar preferences 
with others, this research allows instructors to gain a firmer understanding of different student wants and needs. 
Most importantly, this research aims to support instructors who feel that online writing courses can be successful 
and can promote just as useful of learning as a face-to-face course does. 

Keywords: Online learning readiness, perceived academic confidence, online and face-to-face comparison, online 
writing course, motivation and learning styles 


Introduction 

Online education is now a household term in the world of education. Students enjoy online courses for a 
multitude of reasons, with some enjoying certain aspects of course design and others enjoying the relative freedom 
online learning allows. For instance many students enjoy the “flexibility, convenience, and time efficiency” that 
online learning affords them (Jaggars, 2013, p. 9). With the ability to be connected all of the time, students can 
travel to campus less, can work a more set schedule (Jaggars, 2013), and online learning allows students to use a 
multitude of technologies that are a facet of their day to day lives. Most importantly, many students find that they 
can utilize class time better (Jaggars, 2013). In her study, Shanna Jaggars of Columbia University Teachers College, 
interviewed a student that said her “face-to-face English instructor. . .keeps talking and talking and fills up the whole 
class, instead of letting us go to work on the papers” (Jaggars, 2013, p. 7). 

Many educators fear that online education does not meet the needs of their students (Sapp & Simon, 2005; 
Peterson, 2001; Jaggars, 2013). Many of these fears stem from an understanding of how courses were delivered in 
more traditional, face-to-face setting in the past (Miller, 2001). While this fear may have been more prevalent when 
online instruction was first beginning, it seems to become less and less of a factor as technology becomes even more 
widely used in instructional settings. Interestingly, some students prefer to take “harder” courses offline and prefer 
face-to-face courses when needing more professor-student interaction (Jaggars, 2013). As such, it’s not only some 
professors who are averse to online learning in certain situations. 

Research Questions 

The following questions were used to frame the data collection process in general. 

1 . Is there a difference in student attitudes and perceptions between the online and offline writing courses? (e.g. 
attitudes about difficulty, how instruction is framed, quality of education, ability to learn through this medium, etc.) 

2. Do the overall perceived student writing efficiency and satisfaction of learning outcomes differ between students 
at the end of their respective courses? 

3. Overall, how satisfied are the students with their learning experience due to their choice of delivery format? 

4. How do a learner’s experiences (with factors such as online learning, technology skills, and perceived writing 
competency) affect their choices of learning delivery format? 

Literature Review 

Determining student attitudes, their content knowledge mastery, and student preferences of writing courses 
are of great importance to the larger educational community. The importance of knowing the above becomes even 
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more important when one reviews how popular online learning has become. In their report of online learning for 
the Babson Research Group and the Sloan Consortium, Allen and Seaman (2010, p.5) found that there was a “total 
of 4.6 million online students” in the Fall of 2007, a total that increased by nearly 500,000 in four years (Allen & 
Seaman, 2011). 

Currently, many scholars are leaning towards blended delivery formats in terms of course design. This is 
due to the higher attrition rates in fully online programs (Rovai & Jordan 2004; Perry & Pilatti 2011), a better sense 
of community in blended programs, and their ability to allow some face-to-face time for students and faculty 
alike. Blended courses can deliver the positive aspects of both fully online and offline courses, yet still allow for 
some face-to-face instruction. 

Learner Attitudes 

Student attitudes towards online learning, and their course, are an important aspect of the learning 
process. Knowles and Kerkman (2007), noted a host of attitudes that are facets first-time and returning online 
learners’ experiences. For instance, many students “expect online courses to take less time since they don’t have to 
sit in a classroom for a specified amount of time.” (p. 76). The two also reported that students found it challenging to 
review online materials, yet students were actually able to review more information than they could in a face-to-face 
class. Interestingly, Palmer and Holt (2009) found that students share many of the same attitudes about online 
courses as they do offline courses. For example, students in online classes “were most concerned about.. .what they 
need to know/do to get a good mark/grade and receiving useful feedback on their assignment work.” (p. 109) 

Andrew Morozov explored face-to-face writing courses and student attitudes towards them. Morozov 
surveyed undergraduate students about their attitudes regarding the “writing tasks in writing-intensive courses” 
(2010, p. 7). Students felt they needed timely feedback, clear objectives, and prefer overall course clarity (Morozov, 
2010). As Palmer and Holt found above, learners tend to share similar attitudes of courses regardless of being 
online or offline. As such, online students most likely have the same attitudes about what makes a good online 
writing course. Unfortunately this can’t be certain because many studies do not state writing course (offline or 
online) student attitudes (Newell et ah, 2011), which this study explored. 

Many attitudes are developed based upon what students think of online courses after taking one. Sun et ah, 
in their meta-analysis of e-learning practices, found a slew of factors in what makes e-learning successful for the 
vast majority of students. These factors included easy-to-navigate course design, a student’s level of technological 
knowledge/skill sets, instructor’s ability to use the medium well, and a combination of easy-to-use, yet diverse 
learning options and assignments (Sun et ah, 2008). When looking at any study on online education, one has to be 
careful because the effective practices will vary between course type, the institution, and the students taking the 
course (Means et. al, 2010). However, Sun and other researchers have begun to slowly build up a base of 
commonalities for educators to review before designing their own courses. 

Anxiety 


Anxiety for college level academic writers can have an overall effect on their success (Moore & Kearsley, 
2011). Due to the fear of learners in being successful in their respective writing course, and degree program as a 
whole (Pitma et. al, 2009), students will sometimes shy away from their authorial identity to instead follow practices 
of other writers (Pitma et. al, 2009) or their instructors. Students who feel anxious can “demonstrate behavioral 
anxiety through avoidance, withdrawal, and procrastination in completing their writing assignments” (Martinez, 
Kock, Cass, 2011, p. 352). As shown above, this can negatively affect course success and for students to not gain 
the academic writing skills that they need to succeed in their academic career. 

Much like students can face anxiety over their writing, students can face anxiety over taking online courses 
themselves. (Moore and Kearsley, 2011) Moore and Kearsley found that usually the aspect of an online course that 
most affects a student’s anxiety is the course instructor. (2011) In a study conducted by Dianne Conrad, a professor 
at the University of New Brunswick, students used adjectives to describe how they felt towards online 
instruction. Males and females described feeling “anxious, apprehensive, cautious, scared, intimidated, vulnerable, 
trepidation, [and] terrified” (Conrad, 2002, p. 209) towards online courses. Meanwhile, some students felt 
conflicting emotions like excitement, yet fear still was an aspect of their feelings toward their upcoming coursework 
(Conrad, 2002). 
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Student Perception Factors 

A common thread in the literature is that students, of all grade levels in higher education, want to feel 
connected to their university through an ingrained sense of community. Exter et al. mentioned that program wide 
community may be important to establishing connectedness, based on their review of relevant literature (2009). The 
lack of community, among many other factors, may be an influence to the high attrition rates of online-based 
programs (Exter et al., 2009; Rovai & Jordan, 2004; Perry & Pilatti, 2011). In fact, building a sense of community 
and student interaction “is strongly related to online learning enjoyment, effectiveness of learning online, and the 
likelihood of taking another online class.” (Muilenburg & Berge, 2005, p. 45). 

Student’s perceptions of support also affect how accurately students feel they are meeting learning 
outcomes (Lee et. al, 2011). The more students feel supported by their instructor, the more students increase their 
feeling of success and fulfillment of course outcomes (Lee et. al, 2011). Interestingly, learning outcomes and goals 
can also be supportive to the student, helping them in “preparing for exams and completing course projects, 
especially in online courses, where we do not meet face-to-face to elaborate on course goals” (Savenye, Olina, 
Niemczyk, 2001, p. 375). However, other researchers have found that online writing learning outcome, and the 
success of students in meeting them, will be dependent on the group of students, even if the course remains 
unchanged (Miyazoe & Anderson, 2009). 

Self-Efficacy 

Self-efficacy can be a major aspect of one success as a college writing student. Pajares found that 
“research findings have consistently shown that writing self-efficacy beliefs and writing performance are related” 
(2003, p. 144). Pajares concluded in his research study that self-efficacy, or a student’s belief that they can write 
well (and in this academically well), is important to their overall course success and motivation to learn writing as a 
whole (2003; Pajares, 1996). Consequently, it is realistic that online learners may have another added need for self- 
efficacy in terms of their feelings of their own abilities to succeed in an online setting. 

Online learning also demands students to be more self-reliant and self-motivated in nature if they are to be 
successful (Perry & Pilati, 2011; Nakayama & Yamamoto 2011; Rovai & Jordan 2004; Muilenburg& Berge 
2005). This self-reliance is not only focused on schoolwork, but on technical capabilities as well. Students who are 
not as technologically skilled members of the digital divide (Snart, 2010) face an added layer to their 
schoolwork. Not only are they working on schoolwork, but also many times they are learning how to email, use 
instant messaging programs, bulletin boards, etc. These students may “react to new educational technologies with 
varied emotions, ranging from enthusiasm to disabling fear” (Rovai& Jordan 2004, p. 2). 

The Department of Education found that “online learning has been modestly more effective, on average, 
than the traditional face-to-face instruction with which it has been compared” (Means et al, 2010, p. 51). However, 
many of the positive studies of online learning actually involved blended learning practices (Means et al, 2010). As 
such, some form of face-to-face interaction amongst students and faculty seems needed. As time continues, more 
students will be technologically savvy and likely will want to take online courses. Our world is more wired than 
ever before and education is following suit. 

Summary 

In summary, online learning has many advantages and disadvantages in terms of course delivery. Students 
enjoy the fact that there are no face-to-face class hours involved, yet find that they are actually working harder to 
learn course content in an online class. If the student does not feel connected to their university, fellow students, and 
professors, then there was a potential for the student to drop the course or not take online courses in the future. Most 
importantly, students want the same things from an online course as they do from an offline course. They want 
timely evaluation, to learn useful course content, clear learning outcomes, and to be treated as any other student 
would. Lastly, there was a lack of specific online versus face-to-face writing course literature; this study aimed to 
fill that gap in knowledge. 
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Methods 


Research Design 

At a small public university in California, undergraduate students declare their major through taking a pro- 
Seminar course. Each pro-seminar course is split into two sections. The first section is comprised of lectures and 
guest presentations aimed at helping students further delineate their personal and professional goals. The second 
section, the writing lab, is focused on developing writing skills. Only the lab section can be taken online or face-to- 
face, whereas the lecture section is a required face-to-face section of the course. This project investigated four 
writing lab sections, two that are offered fully online and two that are offered face-to-face. 

Students in the lab learned not only how to structure and improve their writing, but also how to make it less 
technical and more understandable for those who are not members of the computer science population. However, 
both sections had the same course assignments, with certain assignments being slightly changed for the online 
section (e.g. asynchronous recorded video). 

Through a pre-survey, mid-semester interview, and post-survey, research data were collected and run 
through a SPSS statistical analysis. Using the Intrinsic Motivation Inventory (IMI) as a basis for developing the pre 
and post survey, and interviewing students of each course for qualitative data, this research helps promote a further 
understanding of student perceived readiness, student attitudes, student perceived ability, and their effects on student 
success within these two writing learning environments. 

Data Collection Procedures 

The pre-survey was sent over Google Forms, with the class professor and researchers informing the 
students of the survey. The link was housed on the university’s Learning Management System (LMS) for students 
to access, and the post-survey used the same method. The researchers gathered the responses into one Excel file and 
used it to run the needed statistical operations, in order to delineate the data further. Comparisons were made 
amongst classes, and amongst students between online and offline to see differences in attitudes, learning style, and 
preferred teaching methods (all of which are assigned a number rating in each survey question on a Likert Scale). 

To expand on the data collected in the surveys, the interview process involved follow up questions related 
to attitudes, learning styles, delivery and teaching methods. The researchers asked a smaller sample size of students 
about their experiences in their respective courses. The researchers tried to get as representative of a sample as 
possible and had five students from an online section of the course and five from an offline section of the course. 
Each interview session was recorded, which allowed the researchers to code responses to find commonly used words 
and phrases. 

The post-survey was given to each class section toward the end of the semester. The post-survey asked the 
same questions as the pre-survey (except for small changes in wording, etc., based on the class coming to an end) to 
see how students' attitudes, learning style preferences, and teaching method preferences were changed. This data 
were compared to the pre-survey data later. 

Participants 

A total number of 95 undergraduate students participated in the study. They varied widely in terms of 
gender, racial background, and major concentration. 73 out of the 95 students were male and 22 of them were 
female students. There were one American Indian, twenty Hispanic/Latinos, seventeen Asian, three Black/African 
Americans, forty-five White/Caucasians, four Pacific Islander, and five other students who identified themselves as 
others. The ages ranged from 19 to 61. Many students speak another language at home instead of English and many 
of the students were bilingual or multilingual. 

Among the survey respondents, fifty-seven percent of the students were face-to-face students (54 total 
students), while forty-three percent were from the online sections (41 total students) filled out the pre-survey. Of 
these responders, sixty-eight students were majored in Computer Science and Information Technology (72%), while 
twenty-seven students were majored Communication design (28%). As such, students have varying technological 
expertise from which to draw from, and an even broader set of writing skills. 

About 62% of the students had taken at least one online writing course, or one writing course before. 
Interestingly, even though many of the students found themselves to be technologically savvy, 58% of the students 
would prefer to take a face-to-face version of the course over an online version. In fact, 68% of students found that 
they learned better in a face-to-face class compared to an online class. 
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Data-Analysis Procedures 

The pre-survey was delivered to the target population in the third week of the semester. This was done for 
two reasons. Firstly, the researchers wanted students to feel comfortable with the online learning management 
system because it had updated from the previous year. Also, this would allow for transfer students to get a feel for 
how the LMS worked and how the campus log-in system worked as well. By doing this, data was less likely to be 
affected by the changes to the LMS. 

The pre-survey included questions customized from the Intrinsic Motivation Inventory. The Intrinsic 
Motivation Inventory (IMI), used in the pre and post survey, was a validated scale created by Richard Ryan in 1982. 
This scale “is a flexible assessment tool that determines subjects’ levels of intrinsic motivation as an additive 
function of the underlying dimensions of interest-enjoyment, perceived competence, effort, and pressure-tension” 
(McAuley, Duncan, Tammen, 1989, p. 49). The scale has been adapted from Likert-scale questions, to include 
multiple choice response questions, and open questions that students can write full responses too as well. This 
version of the scale has been adapted to fit the learners being asked to be a part of this study, and as such, allows for 
the information to be fruitful for our study. 

The survey was created using Google Forms and was housed on each course section’s LMS homepage. 

The students were asked to complete the survey by their professor, as well as the researchers. Due to a consistent 
slate of reminders, the survey had 95 respondents out of a possible 97 students, reaching a response rate of 97.9%. 

The pre-survey allows for collection of both quantitative data and qualitative data. Students assessed their 
academic writing readiness, skills, and past experiences based on a Likert Scale ranging from 1 to 7 and with prompt 
words including “not at all”, “greatly”, etc., to help students display their perceived abilities. After the quantitative 
questions in a given section, the students were asked to offer additional insights about course delivery preferences, 
writing activities they enjoyed, and about past learning experiences. 

The post-survey was delivered during the 15 th week of the semester. The survey was housed on the LMS 
course homepage, in a similar fashion to how the pre-survey was displayed. The researchers and professor of the 
course asked for students to respond. The survey included the same Intrinsic Motivation Inventory from the pre- 
survey with questions. These questions were kept the same to check the changes between each students' response 
from the pre to the post survey. 

The major change between the pre and post survey was the addition of the VARK learning style 
questionnaire (version 7.2). The VARK questionnaire has not been formerly validated in terms of instructional 
design and teaching methodology (Leite, Svinicki, Shi, 2010), however research is pointing to its overall use as a 
way of helping to classify students in terms of where they fall in a learning style spectrum (Leite, Svinici, Shi, 2010; 
Hawk & Shah, 2007; Wehrwein, Lujan, DiCarlo, 2007). Also, the survey asked more open-ended reflection 
questions for students to delve into how their semester went. 

To evaluate the qualitative responses given by the respondents, coding for patterns (Saldalnia, 2009) was 
used. The codes were grouped into primary groups, such as communication, with sub-topics for further delineation 
of the data collected. Also, each code was grouped into either an online or face-to-face category for easier 
referencing. 

The interview questions were constructed based on preliminary data analysis of the pre-survey responses. 
Based on these findings, the researchers designed interview questions that could identify why students felt certain 
ways, and also to determine other relevant information. The researchers asked questions about each student’s 
favorite writing teacher, why students took the section of the course that they did, among others. These questions 
were asked in ways that would triangulate the data collected earlier. 

Results 

An analysis of the Likert scale found differences in a few areas between the online and face-to-face classes 
as a whole. The areas that yielded statistically significant difference includde the enjoyment of writing, perceived 
academic writing skill, personal tension towards academic writing, learning about the academic writing process, 
personal choice when it comes to learning more about academic writing, structured writing activities, working with 
peers and instructors, and the match between a student’s learning style and the course delivery format. 

Student Attitudes (Research Question 1) 

As a class, online students responded that they enjoyed writing outside of school more than face-to-face 
students, (Mean= 3.64 in Face-to-Face compared to Mean=4.0 for Online students). This very likely could be due to 
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online students feeling that they were “good” academic writers (Mean=4.42 in face-to-face versus Mean=4.78 for 
online students). Similarly, online students felt more skilled as academic writers and that their past courses prepared 
them better for academic writing. Consequently, these online students may have felt that their writing abilities 
would allow them to be successful in an online setting. 

The Intrinsic Motivation Inventory used in this study had four categories that learner groups can be 
compared within: choice, interest, pressure, and competence. Negatively worded question ratings were reversed to 
inn statistical analysis. The choice category asked questions in terms of how much each learner felt they had a 
choice in learning more about academic writing. Both groups scored extremely close (face-to-face, Mean=3.31 
mean and Online, Mean=3.4) and were found to have no statistically significant difference in the pre-survey. While 
the mean average range increased in the post-survey, with face-to-face Mean=3.58 and online Mean=3.89, there was 
still no statistical difference. 

Paired sample t-tests were conducted to check the differences in perceived choice within each delivery 
method group. No statistically significant difference was found in the face-to-face sections in terms of perceived 
choice. However, there was a statistically significant difference between online pre-survey scores (M= 3.89, SD 
= .93) and online post-survey scores (M= 3.4, SD= 1.04), (t(40)=2.99, p <.001. ) Online students felt, that they had 
little choice in their academic learning of writing toward the end of the course. Many students, whether in their 
face-to-face interview with the researchers or on short-response questions on the survey, stated that they had to 
read/do course assignments because they were tracked by the course management system. While students liked that 
they had some choice in the topic of their assignments, others wished they had had more choice in the type of 
assignments they could do, e.g. more presentations, or shorter papers. 

The competence IMI questions focused on each student’s perceived academic writing skills and how good 
at writing they perceived themselves to be. There was no significant difference between the online and the face-to- 
face group during the pre-survey. Strikingly, there was no notable difference in means between the face-to-face and 
online groups post-survey. Positively, both group means matched, and went up, showing that most writers felt more 
competent after taking the course. Paired sample tests were run to check the differences in the pre and post 
competence values of each group. There was no significant difference of these in the face-to-face groups. However, 
there was a statistically significant difference in the scores for pre-survey scores (M= 4.27, SD= .71) and post- 
survey scores (M= 4.81, SD= 1.28) conditions (|t(41)|= 3.17, p <.001) in the online groups. While previously, it was 
reported that the online group did not feel that they had much choice in their learning; many did report the 
effectiveness of the coursework. Because online work was tracked, students felt more need to do course readings, 
respond to the classmates in the forum, and reach out to their course professor when needed. Many students also 
found that the assignments helped them to learn more about the topics or to find their weak-points. 

Interest was the third IMI scoring category. This category focused on how interested learners were in 
learning more about academic writing. There was no significant difference between the online and face-to-face 
groups of the pre-survey. Similarly, there was no significant difference in the post-survey scores between the two 
groups. There was no statistically significant difference between pre- and post-survey ratings on this category 
within each group either. 

The final IMI categoiy was pressure. Pressure refers to the tension that a student would feel while 
performing an assignment, learning academic writing, and towards the writing process in general. There was no 
significant difference between the online and face-to-face groups of the pre-survey rating on this item, with online 
showing a mean of 4.1 1 and the face-to-face group showing a mean of 4.1. There was no statistically significant 
difference amongst the groups of the post-survey ratings, with the means widening only slightly from 4.04 online to 
4.26 in face-to-face. There was no statistically significant difference between pre- and post-survey ratings on this 
item with each group either. 

An independent-samples t-test was conducted to compare how nervous students felt when they were 
learning academic writing, between the online and face-to-face conditions. There was a statistically significant 
difference in the ratings for face-to-face (M= 4.58, SD= 1.95) and online students (M= 3.54, SD= 1.72) ratings 
(t(75)= 2.51, p =<.01 ). The questions were negatively worded and were reversed for data analysis. These results 
suggest that online students were more nervous academic writers than face-to-face students. Online students, due to 
lack of constant interaction with the course professor, may well have felt more nervous when performing writing 
assignments. 

Similarly, students were asked how tense they felt while actually performing academic writing for class 
assignments. An independent-samples t-test was conducted to compare the face-to-face and online student ratings. 
There was a statistically significant difference in the scores for face-to-face and online student ratings (|t(75)|= 2.51, 
p < .01). These results suggest that students in the face-to-face sections of the class feel tenser than online students 
do, while writing for their course assignments. 
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Next, students were asked if they felt like they have to learn academic writing. An independent-sample t- 
test was conducted to compare the face-to-face and online student ratings on this question. There was a statistically 
significant difference in the ratings of face-to-face and online students (|t(75)|= 2.79, p < .01). These results showed 
that students who took the face-to-face course felt that they had to learn more about academic writing. Many online 
students stated in the open-ended question that they took the online course largely based on time constraints, and not 
due to whether they actually learned better in this type of format. 

The next follow-up question delved into if a student felt they could learn academic writing online. There 
was a statistically significant difference in the post-survey ratings of this question between the face-to-face and 
online groups (|t(75)|= 2.35, p < .05). The finding suggests that toward the end of the course, the online students felt 
more ready to learn academic writing online than students in the face-to-face writing course. Online students had 
just completed an upper-division writing course online and most likely felt more confident in their abilities to learn 
other subjects in a similar online fashion. Also, many of the face-to-face students mentioned that they felt insecure 
about learning online both in general and in more direct terms with academic writing. 

Delivery Formats and Learning Style (Research Question 2, 3, and 4) 

VARK Scoring 

The VARK learning style questionnaire was included in the course’s post survey, which asked them to 
respond based on a real life scenario to see how they would try to solve the problem. If one response to a scenario 
did not match fully what a student would do, they could circle additional responses to show they could handle the 
situation in a multitude of ways. Each response correlated with a specific learning style of the respondent: either 
visual, auditory, reading/writing, or kinesthetic. 

From the entirety of the student body, reading and writing had the highest average choice with it being 
selected 4.7 times. Kinesthetic learning followed closely with an average of 4.45, with auditory learning selected 
3.7 times, and visual learning selected 3.14 times. From these averages, it can be presumed that a higher average of 
the student learns best reading and writing about a subject and then (or concurrently) practicing the action itself. 

On average, students in the online version of the class relied more on their reading and writing skills and 
their visual skills. Even though many students suggested that they only took the online version of the class because 
of time constraints, many of them had the learning styles that allowed them to be successful in the course. On 
average, face-to-face students relied more on auditory and kinesthetic learning (though the difference was only 
slight on this category). These students often mentioned feedback and interaction with their professor, and fellow 
students, during the free response questions. It should also be mentioned that the differences in the use of 
kinesthetic learning style preferences were slight, with face-to-face students selecting a kinesthetic response to the 
questionnaire more often than online students. It can be surmised that both groups of students highly value and learn 
well from practicing what they have been taught through hands-on exercises, which can be done often in writing 
education. 

In the visual category, there was no statistically significant difference amongst the two learning groups. 
The online group had a higher mean average (3.37 compared to 2.92 for the face-to-face group), yet this could be 
mainly due to the use of visual activities for online learning groups. For example, this group was required to watch 
videos and then report their findings. As such, both groups made use of visual learning, with the online group using 
this learning skill slightly more based on their specific curriculum. 

The auditory category was found to be statistically different. An independent-sample t-test was conducted 
to compare the face-to-face and online group conditions. There was a statistically significant difference in the scores 
for face-to-face and online students (|t(75)|= 2.6, p < .01). One of the face-to-face interviewees shared a poignant 
reasoning as to why this learning style was more prevalent in face-to-face learners. This interviewee shared that 
many teachers will have a PowerPoint, yet they will talk about points that were not on it, or expand on points that 
were on their visual teaching aids. The first interviewee shared that for her, being able to return to visual teaching 
aids in the online section was very helpful for her success. As such, a good auditory learner could feel comfortable 
in a face-to-face course. 

Survey Analyses 

The first open-ended question, which was a follow-up short response to the “do you feel you learn better in 
online or face-to-face courses”, was coded using two keywords: online and face-to-face. After using these two 
keywords, further keywords were found and counted. Pacing, repetition, resources, access to one’s teacher, 
distractions, motivation, quality of curriculum and teacher, feedback, learning style, class setting, relationships, and 
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technology all were highly used words in response to this question. According to a count of responses, 71.6% (58) 
students used keywords that showed a preference towards face-to-face learning, 17.2% (14) students preferred 
online learning, and 1 1 .2% (9) were indifferent towards either delivery method being better for them. These 
responses match closely to their previous response to the two-choice preference question that preceded it. 

Students who preferred online-learning stated that pacing, repetition in face-to-face classes, access to 
further resources, and being able to review material were major benefits of online learning. One student mentioned, 
“writing and lecture courses that were purely based on learning resources and applying that material I am much 
better online going at my own pace”. In fact, nearly every student that mentioned online courses as their preferred 
learning medium mentioned pacing. Pacing, allowed these students to go through course material either quicker or 
at a slower pace than the face-to-face classes, depending on their needs. Many students mentioned that review of 
materials was a great asset of the online section because they could look up additional information if ahead or 
review materials provided by the professor if they preferred taking a slower pace. 

Face-to-Face students found feedback, immediacy of the professor, their learning style, perceptions of the 
coursework itself, and relationships to be the major motivators to take face-to-face courses. Nearly every student 
mentioned the immediacy of feedback and timelier communication as being the main advantage of face-to-face 
classes. Respondents focused on email not being an adequate form of communication because response times were 
inconsistent from professors. Similarly, many students focused on the interpersonal relationships that were formed 
in a more structured setting. Setting itself was another key aspect of the student’s choice of delivery method. One 
student responded “I learn more in face to face because I feel like I have more of a personal experience with the 
professor and because sometimes I find it hard to concentrate while I'm on my computer since I get distracted very 
easily”. Other students found these aspects to assist with networking with both the professor and students as well. 

Those who didn’t prefer a delivery method cited the quality of the teacher and curriculum to be the most 
important factor in how they learn. Others simply felt they would have done well regardless of the course. Lastly, 
others thought it depended on the course content (e.g. a history course, math course) in terms of if they would enroll 
in an online or face-to-face version of the course. 

An independent-samples t-test was conducted to compare how much students liked previous online courses 
that they took with both online and face-to-face student groups. There was a statistically significant difference in the 
scores in the face-to-face group and the online student group (|t(75)|= 2.62, p < .01). This result indicates that 
students who had positive experience with online courses tend to enroll in more online courses. 

Students who preferred face-to-face courses found in-person communication, interaction amongst peers and 
course professor, immediacy of course assignment expectations, and immediate feedback to be important aspects of 
face-to-face courses. Many students commented on the ability to have something further explained in a timelier 
manner and that face-to-face learning was more hands-on in nature. Even more so, these students commented on the 
feeling of community with their peers and a firmer connection with their course professor. For example, one student 
mentioned “with online classes I don't feel that connection with the professor that I do in a face-to-face class. I feel 
like, in person, the professor has a better opportunity to assist student problems and get immediate feedback that is 
not all electronic, but human.” 

Students with a preference for online courses found that this delivery method allowed them to be more 
flexible with the pacing of their coursework and that it required more self-discipline (something commonly harped 
on by face-to-face preferring students). Many of these respondents focused on the lack of time in their personal and 
professional lives, and how traditional classroom courses weren’t right for them. One student, when commenting on 
discipline and time, mentioned: 

“I feel I learn best when I have time to do things on my own. My schedule is hectic and although being in a face-to- 
face environment is nice and mandatory, I am always thinking about doing something else while I am in class, 
which is distracting. 1 have pretty good self-discipline outside of the classroom which allows me to get all of my 
work done.” 

Only five students mentioned writing courses, or academic writing, in their response about preferred 
delivery models. Interestingly, four of the five students believed that writing could be successfully taught in an 
online setting. While one student prefers face-to-face coursework in general, this student responded: 

“1 think my learning is better in a face-to-face class when I don't understand the material or when 1 have to learn 
something new and have questions about it (i.e. computer science or math subjects) With a writing course I know 
grammar, MLA/APA styles (well at least with APA there were enough guides online that can help me), I know 
many words, and I guess writing is more so my thing. So I guess it takes less face-to-face learning in a course where 
I know the basic material than a course where I don't understand the material and would be better learning with an 
instructor giving me examples and hands on activities and help.” 


74 



Other students mentioned the ability to take more time on their writing when they take an online writing 
course. One student commented, “I feel like the writing itself, unless I am learning something new, I do better at my 
desktop computer where I feel the comfort of my own home and I get stuff done in a reasonable amount of time. I 
do the same exact thing when I'm coding.” Naturally other students still preferred face-to-face instruction. One 
student felt that online writing courses didn’t give students the ability to have more controlled writing practice 
opportunities. 


Discussions and Implications 

Through exploring three types of data, one can come to a firmer understanding of how students perceived 
academic writing, their style of learning, delivery formats, and useful instructional practices. Since online learners 
were more nervous about writing, more support should be given to them. A teacher could share that they are 
available in the school chat/email system and offer that their office hours are always available to them. This 
professor did just that, and was also always willing to allow online students to sit-in on the face-to-face section if 
needed. Because the professor did this, students felt supported by the professor and that level of support was 
mentioned in both the survey and interview responses. 

One of the biggest takeaways was that the students who took online classes and enjoyed them would likely 
enroll in future online classes. Teachers, in a needs analysis, should find out if a student was a first-time online 
learner and provide them additional support. Students may feel pressure initially, but if provided support, they may 
find that they enjoy this delivery format. In doing so, learners can once again have a greater choice in how they 
learn. They could pick an online class that aligns closely to their learning style and have a greater chance of 
succeeding at learning the topic. 

Limitations of this study include that the course was an undergraduate course in one department. The study 
included researching how undergraduates were affected by the same online and offline course; however, these 
majors included students who were frequent computer users. For example, a class of Humanities majors might have 
a completely different look on online education. Another limitation lies in the fact that this paper looked solely at 
perceived success, interest, and perceived enjoyment of learning activities. Data, such as grades, course 
assignments, and other hard data could have strengthened the findings presented here. Since access to such data was 
impossible, the researchers could only explore what students though about their writing, instead of how it translated 
to a grade/academic change. Lastly, the researchers would have preferred using a more solidified learning style 
inventory. By using the Kolb learning style inventory, or one that has been researched validated, the learning style 
findings could be more widely understood by the academic community. One major limitation in using the VARK 
learning style questionnaire was that it is multi-modal in nature. Due to this, a student can’t be necessarily defined 
as a visual learner, etc. While this multi-modal approach was correct in terms of showing how every learner was 
truly different, it makes it hard to quantify what type of learner a student most matches. 
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Abstract 

Research shows that there is a direct correlation between the betterment of health and wellness and 
improved grades (Thompson, 2014). This study makes the deduction that student persistence (to completion) is a 
secondary result of the betterment of health and wellness leading to those improved grades. This pilot proposes to 
provide students with wearable technologies and analytics as a means to monitor and improve their physical activity 
in order to improve educational persistence and grades. 

Introduction 

Online student persistence and completion ebbs and flows dependant upon life circumstances. According 
to Moore and Kearsley (2005), “A variety of extracurricular concerns — such as employment (e.g., job stability, 
workload), family responsibilities, health, and social interests — can adversely affect completion of distance 
education course.” The word “persistence” for this study is defined as student class participation through until the 
last week of the class. Hart (2012) further explains, “Terminology has wavered between persistence and success, 
where each has been interchangeably used to characterize a student that completes a course and continues to 
program completion.” 

There is a direct correlation between educational advancement and salary (Carlson & McChesney, 2014). 
With increased focus on performance-based funding there is also an ever increasing need to improve student 
persistence from the educational administrative point of view (Klein, 2015). Therefore, it is critical for both students 
and educators to decrease all barriers to learning in order to increase student persistence, thereby advancing student 
quality of life and increasing educational funding. 

Huang et al.(20 14), Aberg etal. (2009), Centers for Disease Control and Prevention (2010), Erickson 
et al.(20 11) elaborate that, “Evidence has demonstrated positive effects of physical activity and exercise on brain 
structure and cognitive function in humans.” 

Purdue University’s research reflects that students attending their recreation center demonstrated positive 
performance in the classroom. Those who were actively engaged in physical fitness were more likely to succeed in 
their coursework (Neubert, 2013). Harvard University professor extends this study to detail the literal physical and 
chemical linkage between exercise and the brain in his book. Spark (Ratey, 2008). Physical activity is not just a 
“feel good” idea, it literally triggers the brain cells to function at their peak. Dr. Ratey reveals from everyday life to 
the effect on alzheimer patients: “Neuroscientists have just begun studying exercise’s impact within brain cells — at 
the genes themselves. Even there, in the roots of our biology, they’ve found signs of the body’s influence on the 
mind. It turns out that moving our muscles produces proteins that travel through the bloodstream and into the brain, 
where they play pivotal roles in the mechanisms of our highest thought processes.” 

Recognizing the radical growth of online learning for the adult population and the stresses which are innate 
to the many roles these students maintain, supporting their health and wellness is crucial to success (Thompson & 
Porto, 2014). An Oral Roberts University Online press release in the fall of 2014 rolled out its Vivofit activity 
tracker initiative to “support students’ aerobic and fitness outcomes as the university works with them to develop 
mind, spirit and body.” 


Conclusion 

If we provide wearable technologies to students to measure health and wellness, will it increase their 
activity and, thus, student persistence? This study proposes to address this question by providing wearable 
technologies to drive student success during the 2015 year. Indian River State College Adult Education students who 
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are interested in participating will be welcomed to an orientation session which will introduce the benefits of health 
and wellness upon performance and productivity. 

The hypothesis is that students with analytics documenting high level of activities will also demonstrate 
increased alertness and persistence in their classes, ultimately leading to student completion success. 
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Objective 

Based on observed evidence and reports of dissatisfaction by students regarding a computer science course 
focused on encouraging and equipping students seeking teacher certification to incorporate technology into their 
future classrooms, the course in question was redesigned using the ARCS model of motivational design. This model 
focuses on incorporating the tenets of attention, relevance, confidence, and satisfaction into the assignments, 
activities, materials, and delivery of a course (Keller, 2008). This model was chosen because the intent behind the 
course was to empower teachers to effectively incorporate technology into their future classrooms, in the hope that 
this knowledge would position teachers to adapt to the rapidly changing nature of their future classrooms as well as 
increase student participation and interest in the content being taught. It was hypothesized that the application of the 
redesigned course would result in students reporting high levels of motivation to incorporate technology into their 
future classrooms as well as reports of experiencing high levels of attention, relevance, confidence, and satisfaction 
upon completion of the course. This paper focuses on the initial quantitative findings from pre-test/post-test 
research on the redesigned course and provides some qualitative contextualization based on semi-structured 
discussions with students. These findings are presented in order to highlight the potential for future research in this 
area and provide insight into unexpected issues and successes identified by this research. 

Foundations 

The design model chosen to guide the redesign of the pre-service teacher technology course was the ARCS 
model. The ARCS model posits that efforts should be made to incorporate attention, relevance, confidence, and 
satisfaction into the various aspect of a course. Attention was incorporated into the redesigned course via a focus on 
open-ended discussions and varying delivery methods both within a class session and throughout the course 
semester. The course was designed with more direct instruction at the start of the semester to provide more support 
as well as a classroom environment that students were more familiar with. As the course progressed however, the 
course relied more on self-paced exploration of technology, discussions, group/peer support, and increased 
flexibility in assignment focus and outcomes. Each class also featured variations between tool/skill demonstrations, 
open-ended questioning, tool exploration, and question/answer opportunities. Relevance was incorporated into the 
redesign by encouraging students to explore technology and find ways that it could support their desired future 
through in-class activities and assignments. In particular, students were not all required to complete the same 
assignment in the same way as other students. Students were required to demonstrate similar technical abilities, but 
the tools, techniques, design, and content used/incorporated into the assignments were uniquely relevant to the 
individual student. Confidence was built into the redesign by organizing the course in a manner that allowed the 
students to progressively build upon acquired knowledge and skills to achieve tasks that may have initially been 
viewed as unobtainable. Smaller intermediary assignments and activities were used as building blocks and skill 
acquisition opportunities in order to create a final over-arching and summative course project that could be directly 
used in a student’s future classroom. Satisfaction is closely tied to confidence and was integrated into the redesign 
through opportunities to experience achievement and supportive feedback. 

Research Implementation 

A pre-experimental design featuring pre-test and post-test data collection was chosen for this study in order 
to determine if the results of the treatment, the ARCS based pre-service teacher technology course, warranted an 
attempt to conduct a classic experimental design with a control and treatment group. A pre-test/post-test approach 
was taken in both the quantitative and qualitative data collection process. Quantitative data collection was 
conducted through the use of an online survey. 

The setting for the treatment in this study was a traditional computer lab/classroom featuring rows of chairs 
and desks with desktop computers at each chair. The room also included an instructor station with a computer and 
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attached projector and screen. The treatment was applied to two sections of the pre-service teacher technology 
course. One section met twice a week in the afternoon for an hour and a half and the other section met once a week 
in the evening for three hours. This differences between the two sections made a quasi-experimental design 
problematic and supported the decision to conduct initial pre-experimental testing in order to make the case for later 
quasi-experimental testing. 

The data for this analysis was collected from pre-service teachers enrolled in the redesigned technology 
integration course at a medium sized institute of higher education. The population from which the data was 
collected consisted of both traditional and non-traditional students from a variety of backgrounds. The majority of 
participants were female. Participation in the data collection process was voluntary and data collection was 
conducted both in the normal classroom setting and via online surveys. 

The initial pool of respondents included students ranging in age from 18 to 60 and was primarily female. 
Historically the course has only included a few male participants each semester. Enrollment in the course is open to 
all university students and so, while the course is targeted toward education majors, some non-education majors may 
choose to take the course. All students enrolled in the course were invited to participate in the study and were 
notified that no identification information would be collected. In total 64 students were invited to participate in the 
study. Of that 64 only 57 chose to participate in the quantitative data collection process. When the responses were 
examined, it was determined that 19 of the respondents either were not education majors or did not complete both 
the pre-test and post-test. Due to the study design and the focus of the research on motivation by pre-service 
teachers to incorporate technology into the classroom, these 19 respondents were eliminated from the analysis. This 
meant that the sample size for the study was 38. The majority of respondents reported seeking certification in the 
Early Childhood through 6th grade levels and were seeking specialization in areas such as English as a Second 
Language, Bilingual Education, Special Education, Math, and Language Arts. Students seeking certification in 
other grade levels and with other specializations did participate in the study but did not in the majority of responses. 
Of the 38 respondents only one was male. There were five students from each section that volunteered to participate 
in the qualitative data collection. All of these participants were female. 

The Course Interest Survey (CIS) was chosen as the core of the instrument used to collect data in this 
study. The CIS was designed for use in traditional face-to-face, instructor-led educational setting to measure the 
motivation of learners as a combined scale and in the four component areas of the ARCS model in those situational 
motivation focused classroom experiences (Keller, 2006; Keller & Subhiyah, 1993). The targeted purpose of this 
instalment matched the conditions under analysis in this study, and thus the CIS was chosen as the core instrument 
to be used for quantitative data collection. 

The CIS consists of 34 statements related to classroom experiences. Responses are in the form of a 5 item 
Likert-like scale ranging from 1 to 5 which corresponds to a text-based scale of "Not True" to "Very True." 
Evaluation of the instrument was conducted using Cronbach's alpha to determine reliability and correlation analysis 
between student scores and the instrument. The instrument was found to be both reliable and valid. The following 
alphas were obtained during this analysis: .95 for the overall scale, .84 for attention, .84 for relevance, .81 for 
confidence, and .88 for satisfaction (Keller 2006). Several studies reported these same values but did not report if 
confirmatory tests were conducted (Halat, Jakubowski, & Aydin, 2008; Varank, 2005). Confirmatory tests were 
done by one group of researchers and minimal discrepancies in results were found. It was posited that the 
differences may be due to sample size (Huett, Kalinowski, Moller, & Huett, 2008). 

In addition to the CIS survey, additional demographic questions were prepended to the instalment as 
presented to respondents in this study. Questions asking about age, gender, major, specialization, and grade level 
certification were added for the purposes of determining if any of these factors had any bearing on the results. 
Additionally, similar Likert-type items were appended to the instalment to look at the specific motivation of 
respondents to incorporate technology into their future classrooms. The two key items related to the research 
question were: 

• I am motivated to incorporate technology into my future classroom. 

• This course will have a positive impact on my motivation to incorporate technology into my future 

classroom. 

Additionally, four other items dealing with feelings toward technology in a pre-service teacher's future 
classroom were included. These questions were added to form the basis of a potential motivation to incorporate 
technology into the classroom scale. These were: 

• It is important to incorporate technology into my future classroom. 

• I will incorporate technology into my future classroom. 

• I am confident I will be able to incorporate technology into my future classroom. 
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• I look forward to incorporating technology into my future classroom. 

Qualitative data was collected via semi-structured discussions. Notes on student responses were taken at the 
beginning and end of the course for evaluation. Discussion items focused on student experiences related both to 
generalized motivation and the four tenets of the ARCS model. 

Results 

As previously mentioned, similar confirmatory tests for reliability were conducted on the sub-scales of 
attention, relevance, confidence, and satisfaction as well as the overall ARCS scale. These tests were conducted for 
both the pre-test and post-test data collected in this study. Pre-treatment and post-treatment overall scale analysis 
indicated strong internal consistency, evidenced by Chronbach's alphas of .944 and .959 respectively. Sub-scale 
analysis of the pre-test data resulted in the following: .770 for attention, .868 for relevance, .803 for confidence, and 
.824 for satisfaction. Post-test sub-scale analysis yielded the following: .853 for attention, .865 for relevance, .846 
for confidence, and .897 for satisfaction. A summary of these results as well as the original scores obtained by John 


Keller (2006) are shown below. 


Instillment Scale Reliability Values 

Scale 

Original Values 

Pre-Test Values 

Post-Test Values 

Total Scale 

.95 

.944 

.959 

Attention 

.84 

.770 

.853 

Relevance 

.84 

.868 

.865 

Confidence 

.81 

.803 

.846 

Satisfaction 

.88 

.824 

.897 


As can be seen in the table, while some discrepancies in the scores can be seen, overall, the pre-test and 
post-test scale values are in line with the values from the original evaluation of the Course Interest Survey. Of note 
in these findings is the score of .770 in the pre-test attention scale, but this value came in line with the original 
scores when evaluating the post-test data. Furthermore, reliability tests were conducted for the potential motivation 
to incorporate technology scale and a value of .908 was found for the pre-test and a value of .913 was found for the 
post-test. 

It was hypothesized that the application of the treatment in this study would result in a significant 
difference between pre-test and post-test results. An analysis of the pre-test and post-test scores for the four scales 
of attention, relevance, confidence, and satisfaction, as well as generalized motivation found no significant 
differences between these scores. An alpha level of .05 was chosen for these tests. The p-values obtained from 
these tests are provided in the table below. 


Paired Samples t-Test Results 

Scale 

P-Values 

Attention 

.603 

Relevance 

.199 

Confidence 

.164 

Satisfaction 

.100 

Total Scale 

.155 


As can be seen in the above table, none of the p-values fell below the chosen alpha level, indicating that no 
significant differences existed between the scores. Thus, the null hypothesis could not be rejected. Additional tests 
were conducted on the two key questions appended to the CIS instrument used in this study as well as the scale for 
the potential motivation to incorporate technology into the classroom. While no significant difference were found 
between pre-test/post-test scores for the potential motivation to incorporate technology scale (p = .093) and the key 
question dealing with students’ reported motivation to incorporate technology (p = .853), a significant difference (p 
= .026) was found for the following instrument item: 

• This course will have a positive impact on my motivation to incorporate technology into my future 
classroom. 

Further analysis of the means for this item showed that the difference was actually due to a drop in scores 
between the pre-test and post-test. When analyzing the means from the scales and items analyzed in the paired 
sample t-tests, it appears that both the pre-test and post-test scores were above average and that actually a slight drop 
can be seen between pre-test and post-test scores for all of the scales/items except one. The one exception to this 
drop is the respondent’s reported motivation to incorporate technology into their future classroom. However, the 
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rise noted between those means is negligible. This trend is counter to the expected effect of applying the treatment. 
The table below summarizes the means for the various scales a nd items under analysis. 


Scale/Item Pre-Test and Post-Test Means 

Scale/Item 

Pre-Test Mean 

Post-Test Mean 

Attention 

3.69 

3.63 

Relevance 

4.26 

4.13 

Confidence 

4.04 

3.92 

Satisfaction 

3.90 

3.72 

ARCS Scale 

3.98 

3.86 

Potential MTIT Scale 

4.56 

4.42 

Respondent MTIT 

4.45 

4.47 

Course Impact On MTIT 

4.53 

4.18 

Note. MTIT = Motivation To Incorporate Technology 


Upon identification of this unexpected drop in means, tests for normality were conducted on the data to 
determine if paired sample t-tests were appropriate for the data obtained in this study. Because of the small sample 
size in this research, the Shapiro-Wilk test for normality was chosen to analyze the data. This test assumes that the 
data is normally distributed and that p-values below .05 would indicate that there is not a normal distribution within 
the sample and that the null hypothesis could be rejected. A summary of the results of the Shapiro-Wilk test can be 
seen in the following table: 


Pre-Test and Post-Test Sample Normality Test Results 

Scale/Item 

Pre-Test p-value 

Post-Test p-value 

Attention 

.152 

.018* 

Relevance 

.012* 

.007* 

Confidence 

.055 

.002* 

Satisfaction 

.392 

.018* 

Total ARCS Scale 

.094 

.023* 

Potential MTIT Scale 

<.001* 

<.001* 

Respondent MTIT 

<.001* 

<.001* 

Course Impact On MTIT 

<.001* 

<.001* 

Note. MTIT = Motivation To Incorporate Technology 
*p < .05 


As can be seen in the table, only the pre-test samples for the attention, confidence, satisfaction, and total 
ARCS scales appear to be normally distributed. These findings indicate that parametric tests may not be appropriate 
for this data because the majority of samples appear to be abnormally distributed. The Wilcoxon signed rank test is 
an alternative to traditional paired sample t-tests when the data is not normally distributed, and so this test was 
conducted on scale and item scores. The results of the W ilcoxon signed rank test are shown in the table below. 


Non-parametric Related Samples Test Results 

Scale 

P-Values 

Attention 

.911 

Relevance 

.312 

Confidence 

.213 

Satisfaction 

.128 

ARCS Scale 

.219 

Potential MTIT Scale 

.063 

Respondent MTIT 

1.000 

Course Impact On MTIT 

.027* 

Note. MTIT = Motivation To Incorporate Technology 
*p < .05 


These findings mirror those found using paired sample t-tests. No significant difference was noted between 
pre-test and post-test scores except in the case of respondents’ reported belief about the impact of the course on their 
motivation to incorporate technology into the classroom. The null hypothesis in this research was that the treatment 
did not have an effect on student’s motivation to incorporate technology into the classroom. The results of both the 
parametric and non-parametric tests conducted help confirm the acceptance of this null hypothesis. The one 
exception to this being the survey item concerning the positive impact of the course on pre-service teacher 
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motivation to incorporate technology into their future classroom. While demographic data was collected in order to 
analyze demographic factors in relation to the scales and items, it was found that several of the demographic sub- 
groups only contained one or two respondents. Because of this skewed data and the small sample size, analysis 
between demographic factors and the scales/items was unreliable. Additional data collection to increase sample size 
would need to be conducted in order to analyze these factors. 

Qualitative data was analyzed via researcher reflection and interpretation. Key themes identified in this 
process were: technology incorporation fear/fear reduction, perceived technology incorporation barrier reduction, 
and technology as an inclusionary tool. A coding and confirmatory analysis was conducted to support the themes 
identified in the reflection and interpretation process. In both the pre-test and post-test discussions there was a 
generally positive motivation to incorporate technology into the classroom, but post-test responses tended to be 
more positive and include more examples of how respondents would incorporate technology. Regarding the ARCS 
tenets of motivation, in the pre-test, respondents had high expectations that these factors would be present in the 
course but lacked specific examples of how they would manifest themselves. In the post-test responses, the 
experiences tended to confirm the expectations, but specific examples of the four factors were shared. These results 
were mirrored in student responses related to the impact of the course on their motivation to incorporate technology. 

Discussion 

This research focused on the quantitative results obtained from an analysis of pre-test/post-test data 
collected from students enrolled in a redesigned course designed to equip and encourage pre-service teachers to 
incorporate technology into their future classrooms. The course was redesigned using John Keller’s ARCS model 
of motivational design as it appeared to be a “best fit” for the course due to the intended purpose of the course. It 
was hypothesized that using this model as the basis for the redesign of the course would result in reports of high 
levels of motivation to incorporate technology into their classrooms by students who had completed the redesigned 
course. Additionally, it was hypothesized that students would report experiencing the ARCS components of 
attention, relevance, confidence, and satisfaction upon completion of the course. Data was collected using a pre- 
test/post-test model to discern potential changes in participant responses due to the treatment course. 

Confirmatory reliability tests of the scales present in the Course Interest Survey, the core instrument used in this 
research, were conducted to compare the results from this study with those found by Keller (2006). The results of 
these tests were mostly in line with Keller’s original findings with some discrepancies. These variations may have 
been a result of the small sample size analyzed in this research. Similar variations have been identified by other 
researchers (Huett et al., 2008). 

Parametric and non-parametric tests for significance between pre-test and post- test scale/item scores 
indicated no significant difference except in the case of an appended instrument item concerning the positive impact 
of the course on student motivation to incorporate technology into the classroom. Further analysis of the means 
showed that in the case of this item, as well as in the case of the majority of the other scale/items where no 
significant difference was found, a drop had occurred in the scores between the pre-test and post-test responses. 

This was counter to the hypothesized effect of the course on student responses. 

These results may be due to a potential ceiling effect. The pre-test quantitative data as well as the pre-test 
discussions conducted with students indicated that students entered the course with high levels of motivation to 
incorporate technology into their future classrooms. Furthermore, students indicated high expectations of the course 
to motivate them to incorporate technology and to address the four components of the ARCS model. The high 
levels of motivation and expectations seen in the pre-test quantitative data may mean that any effect the course may 
have had could be masked by the high pre-test scores. This potentiality is further supported and contextualized by 
the qualitative data collected in this research. In discussions with students about motivation to incorporate 
technology into the classroom, students reported being highly motivated and expected the course to motivate them. 
While students reported these expectations and motivation in the pre-test, they were unclear as to how the course 
would meet these expectations and how they would incorporate technology into their future classrooms. Post-test 
discussions revealed a greater understanding of how the course motivated them and specific ways in which they 
would incorporate technology into the classroom. The mean comparisons seen in the quantitative findings and the 
contextualization provided by the discussions indicate that a potential ceiling effect needs to be investigated. 

The qualitative results also help to explain the significant difference, a drop in scores, seen between the pre- 
test and post- test data concerning the impact of the course on motivation to incorporate technology. At first glance, 
the quantitative data may indicate that the course had a negative impact on motivation to incorporate technology. 
However, the responses from students in the discussions hints that this quantitative finding may be a case of the 
course failing to meet the high expectations students had of the course before actually experiencing the course. 
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Additionally, tests for normality, indicated that the sample may be abnormally distributed, which could lead 
to unreliable results when analyzing the pre-test and post-test scores. While non-parametric tests conducted to 
address the lack of normality served to affirm the results found in paired sample t-tests, the failure of the samples to 
meet the normality tests coupled with the small sample size, and skewed demographics, indicate that further 
research needs to be conducted with a larger sample size to address these issues. That said, the data for this research 
is being collected from a situation that tends to result in uncontrolled and significantly varied sample loading. This 
coupled with the varied background experiences of enrolling students may continue to result in abnormally 
distributed samples. 

The small sample size, abnormally distributed data, and pre-existing expectations of the course serve as 
limitations to the findings reported in this research. While the quantitative data indicates that the course did not 
have an impact, or potentially had a negative impact, on pre-service teacher’s motivation to incorporate technology 
into their future classroom, the contextualization provided by the qualitative results indicate the potential ceiling 
effect and aforementioned limitations of the findings indicate that a potential effect could still exist and that 
additional research needs to be conducted. Specifically, larger sample sizes could help reduce several of the 
limitations to these findings. Early testing, perhaps upon initial enrollment into the teaching program may be a 
better point at which to conduct pre-testing in order to reduce the effect of high expectations of the course on the 
scale and item scores and reduce the potentiality of a ceiling effect. Furthermore, the inconclusive nature of the 
findings from this pre-experimental testing do not necessarily negate the need for quasi-experimental testing. 

Instead, they support the need to conduct quasi-experimental research with more closely matched sections of the 
course in order to address some of the limitations addressed in this paper and gain better insight and understanding 
into the effect of the redesigned course. 
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Introduction 

Given the recent rapid innovation of e-learning authoring tools, academic programs responsible for 
preparing future training practitioners are faced with the difficult task of deciding how best to design curriculum for 
e-learning production skills for aspiring instructional designers and multimedia developers. To be able to 
appropriately design curriculum in academic programs, it would be valuable to know what tools experienced e- 
learning designers and developers frequently learn to use and how they select specific tools. This is important 
because, although many authoring tools advertise similar functionality, experienced instructional designers (IDs) 
and multimedia developers understand that there are subtle differences that need to be considered for novice and 
experienced users when thinking about the needs for a specific project. The motivation behind this pilot study is 
from students in a graduate e-learning certificate and master’s degree programs consistently asking professors for 
support to learn how to use and select multimedia-authoring tools. We turned to the literature to gain insight on 
evidence-based practices to help meet this need; however, the research in this area is currently barren. 

Selecting appropriate multimedia development software tools 

It is posited that the consideration of the capabilities of different e-learning authoring tools available and 
appropriate selection criteria are typically based on practitioners’ experience (Sweller, 1999; Tyler-Smith, 2006). 
This method of selection is typically problematic for novice instructional designers to choose the most appropriate 
development tool, especially when they have very little time or understanding of what each tool can do for them 
(Hardre, Ge, & Thomas, 2006). Students keep asking what software tools do employers most use and how should 
they become competent in their use. 
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Building instructional design competency and expertise 

There is a small but growing body of scientific literature on strategies and models that can be leveraged to 
help build individuals’ instructional design competency (Bannan-Ritland, 2001; Bichelmeyer, Boling, & Gibbons, 
2006; Carr-Chellman, 1999; Clinton & Hokanson, 2012; Hoadley, & Cox, 2009; Klein & Fox, 2004; Quinn, 1994; 
Shambaugh & Magliaro, 2001). In addition, other researchers are focused on how novice or experienced 
instructional designers learn to develop their practice (Ertmer, York, & Gedik, 2009; Yanchar, & Flawkley, 2014). 
Much of the literature in these areas suggests real world, authentic, project-based experience as necessary for 
students to be able to translate theories and principles into practice in instructional design projects. The science 
behind designing effective, efficient, and engaging instruction is a primary focus of the curriculum, rather than 
software applications. 

Building multimedia software tool competency 

Outside of the instructional design research literature, there are principles identified to support students’ 
software tool competency development. Bhavnani, Peck, and Reif (2012) have described scientific research 
conducted on strategy-based instructional principles for efficient and effective use of computer applications. Further, 
it is common for graduate-level students learning advanced statistics to encounter integrated instruction on how to 
use specific statistical software packages such as SPSS or R to work through problem sets when learning theoretical 
concepts (Mills, 2002). There are published statistical problem set resources and software tutorials readily available 
from major publishers to support these instructional needs. Flowever, we were not able to identify any similar 
research or published resources for multimedia development tools. 

Purpose of study 

The purpose of this study in progress was to investigate how novice and experienced e-learning course 
designers and developers both learn to use, and then select the most appropriate widely available e-learning 
authoring software tool for individual project needs. It is an ongoing research project, for which we have collected 
preliminary data scientific data from a representative sample of the population who have to make instructional 
design decisions based on select tools. 

As a long-term outcome, it is our goal to leverage the impending results from this study to create and 
disseminate a practical set of guidelines. These guidelines would serve as a foundation for those who need to train 
novice instructional designers and educational multimedia developers to develop competency in current software 
tool use and tool selection. In addition, they would provide novice instructional designers with an accessible frame 
of reference to use when selecting tools that would best align with the needs of a given instructional project. 
Research Questions 

During this study, we sought to answer the following research questions: 

1 . What competencies do individuals, who are enrolled in degree programs, believe they need to 
develop? 

2. What e-learning authoring tools are most often used to develop e-learning? 

3. Flow do e-learning instructional designers and developers select different authoring tools? 

4. What training resources do users leverage when learning how to use e-learning authoring tools? 

4.1. Flow does the amount of prior work experience relate to learning methods selection? 

4.2. Flow do competency levels relate to learning methods selection? 

Method 

Participants 

Participants were recruited for this exploratory pilot study from an instructional design listserv managed by 
a small private mid-Atlantic university and from those enrolled in online e-learning design courses offered at a mid- 
sized public Northwestern university. These schools’ programs were selected as pools to recruit our target sample 
participants due to the inclusion of e-learning development courses offered in their graduate program. IRB approval 
was granted for the study at both institutions. 

According to CNN Money (2012) there are 217,700 people working as instructional designers. According 
to the Bureau of Labor Statistics (BLS) Occupational Flandbook does not have a category for Instructional Designer. 
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The Bureau of Labor Statistics (BLS) Occupational Handbook does have a category for Training and Development 
Specialists listed under business and financial jobs. According to the BLS (2012) there were 228,800 Training and 
Development Specialists jobs in 2012. These positions require the functions widely known as part of instructional 
design; including the analysis, design, development, implementation, and evaluation, of training. This suggests a 
population may include up to 228,800 instructional designers and multimedia developers. 

We were able to collect 83 complete data sets from a total of 400 invited participants, for a complete 
response rate of 20.75%. Visser, Krosnick, Marquette and Curtin (1996) showed that surveys with lower response 
rates (near 20%) yielded more accurate measurements than did surveys with higher response rates (near 60 or 70%). 
Further, a population of 288,000 individuals, a 95% confidence interval, and 10% margin of error, would require 80 
sample respondents. Therefore, we believe our data to be an accurate representation of individuals who are 
associated with the position of an instructional designer with responsibilities associated with the design and 
development of multimedia products. 

Procedures 

A web-based survey software application was used to gather self-reported behavioral or skills responses for 
a range of items types, including: sixteen multiple choice, thirty-six rating scales, and fourteen open-ended 
questions. The survey was sent via email to individuals who were subscribed to both universities’ listservs that serve 
instructional designers and instructional design students. Participants noted their informed consent prior to any data 
collection. 

A mixed methods concurrent nested strategy was implemented for data analysis. The quantitative data was 
analyzed using frequencies and descriptive statistics, including measure of central tendency such as mean and mode. 
Correlation analyses were used to uncover relationships between variables such as experience, competency, and 
approaches to learn to use e-learning authoring tools. The qualitative data was then analyzed to further explain the 
quantitative findings and answer the research questions about how novice and experienced users select authoring 
tools. 


Results and discussion 

Research question 1 : What competencies do individuals, who are enrolled in degree programs, believe they need to 
develop? 

Over half (57.80%) of the survey respondents were enrolled in an instructional design or workplace 
learning degree program. The data revealed that participants’ enrollment status has a small to moderate significant 
correlation between their enrollment and ID competencies. Those who were not enrolled in a masters’ program 
reported more competency with evaluation, project management, and implementing the ADDIE process. Clearly 
individuals enrolled in master’s degrees are working on developing competencies that are foundational to e-learning 
design and development projects such as storyboarding, working with SMEs, writing objectives, assessment items, 
and creating e-learning. These results are shown in Table 1. 

Table 1 

Correlations between enrollment status and ID competencies 


Write 



Evalua- 

tion 

Project 

Mgmt 

ADDIE 

Story- 

board 

Work w / 
SMEs 

Write 

Objectives 

assessment 

items 

Create e- 
learning 

Spearman’s 

Correlation 

Coefficient 

0.292 

0.258 

0.342 

-0.318 

-0.419 

-0.164 

-0.234 

-0.327 

Sig. 

0.01* 

0.021* 

0.002** 

0.005** 

0.0** 

0.138 

0.035* 

0.003* 

N 

77 

80 

83 

76 

83 

83 

81 

81 


Note. Enrollment status was coded as yes (1) or no (2). ID competencies were reported on a scale of 1 - 
5, from not competent to very competent. 

*Correlation is significant at the p < 0.05 level. 

**Correlation is significant at the p < 0.01 level. 
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Research question 2: What e-learning authoring tools are most used in the field to develop e-learning? 

Table 2 shows the percentage of the respondents who reported having used of each tools mentioned above, 
the mode response for each tools’ users’ total years of experience, ability level ranging from novice, to intermediate 
to expert, and a mean rating number based on a scale of one to five. Participants rated each tool according to ease of 
use, meeting needs, and access to help, with one being the least satisfactory rating and five being the greatest 
satisfactory rating. We allowed three additional opportunities for participants to enter other tools used, which were 
not identified on the list below. Flash was reported as being used by a few respondents. However there were not 
enough people using it, to signal an importance to categorize it and study its use as an e-learning authoring tool. 

Table 2 


Summary of tool use, years of experience, ability level, and rating 


Tool 

Use 

Experience 

Ability 

Rating (1-5) 

Articulate 

70.37% 

1-7 Years 

Intermediate 

4.2 

Camtasia 

60.24% 

1-7 Years 

Intermediate 

3.8 

Captivate 

73.17% 

1-7 Years 

Intermediate 

4.4 

Lectora 

28.91% 

< 1 Year 

Novice 

3.9 

PowerPoint 

98.80% 

10+ Years 

Expert 

4.6 


Articulate was a tool that 70.37% participants reported having used to develop e-learning software. While 
31.57% of the respondents had less than one year of experience, 59.65% had one to seven years of experience. Most 
people who use Articulate are not experts. Camtasia was a tool that 60.24% participants reported having used this 
tool. While 38.00% of the respondents had less than one year of experience, 48.00% had one to seven years of 
experience. Most people who use Camtasia are not experts. Captivate was a tool that 73.17% of the survey 
participants reported having used to develop e-learning materials. While 25.00% of the respondents had less than 
one year of experience, 55.0% had one to seven years of experience. Most people who use Captivate are not experts. 
Lectora was a tool that 28.91% participants reported having used this tool. While 41.67% of the respondents had less 
than one year of experience, 50.00% had one to seven years of experience. Most people who use Lectora are not 
experts. 

Nearly all (98.80%) survey respondents reported having used PowerPoint to develop e-learning materials. 
While 2.43% of the respondents have less than one year of experience using this tool to create instructional 
multimedia, 14.63% had one to years of experience using this tool to create instructional multimedia, 69.51% had 
more than ten years of experience using this tool to create instructional multimedia. Most people who use PPT to 
develop e-learning are expert users, where expert is someone with ten years of experience. 

We used Spearman’s correlation coefficient to further investigate relationships between tools use because 
we did not have a normal distribution for participants’ experience in their current role and experience with each of 
the tools. We found that those who tend to use Captivate also tended to have experience using Articulate in their 
work, as evidenced by the moderate positive correlation between use of Captivate and Articulate, r,(82) = .475, p = 

< .001. Those who tend to use Lectora also tended to have experience using Camtasia in their work, as evidenced by 
the moderate positive correlation between use of Lectora and Camtasia, r,(82) = .451, p = < .001. This relationship 
suggests that it may be helpful to guide students towards developing skill sets for more than one tool. Also, 
Articulate and Captivate are the two most often used tools and may increase an individual’s employability. 

Research question 3 : How do e-learning instructional designers and developers select different authoring tools? 

Qualitative coding revealed three overarching themes and ten corresponding categories, each with multiple 
entries. The reasons that e-learning authoring tools are selected center on the comfort level of the individuals who 
need to use the tool, the tool’s availability, and compatibility with the project. Each of these themes along with their 
corresponding categories, frequencies, and examples are shown in Table 3. 
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Table 3 


Summary of themes, categories, frequency, and examples, found in open ended responses to reasons for 
e-learning authoring tool selection 


Theme 

Category (occurrences) 

Frequency 

Examples 

Designer or 
developer 

Easy to use 

17 

“Ease of use...,” 

“Because it was easy to use...” 

comfort 
level with 
tools 

Prior experience with tool 

Developer community 
support 

4 

3 

“Because it was the only one I knew.” 

“It's the one I know.” 

“The online community is awesome. Have a 
problem or need a template? Check out the 
community. Plus the tool now comes with 
templated objects such as characters.” 

“Seemed to have a great community of 
support...” 

Tool 

availability 

Client request 

33 

“Company requirement...,” 

“Selected by customer.” 


Own license 

12 

“Company owns license.” 

“Earlier version licenses had been purchased.” 


Cost 

7 

“Grant-funded.” 

“...cost of the package.” 

Tool 

compatible 
with project 
scope 

Product matched training 
need 

16 

“I selected PowerPoint because I didn't need the 
screen capture abilities of Captivate as the online 
module is simply informative and used to 
socialize something new.” 

“It was the best fit for what the instructor was 
trying to achieve.” 


Compatible with system 

11 

“...Works well on our learning management 
system.” 

“...integration with our LMS.” 


Fit with content 
requirements 

10 

“The type of content could be best delivered after 
published in this tool.” 

“Because of audio ability and screen recording 
ability.” 


Time 

4 

“Required less time...” 

“Time to develop” 


Research question 4: What training resources do users leverage to learn how to use e-learning authoring tools? 

As we know, novice learners often learn more efficiently when experts provide guidance and structure to 
problem or task centered learning needs. However, most respondents who need to learn to use e-learning authoring 
tools do so without the help of what one might think of as formally structured training and expert guidance. It 
appears as though most people are learning the software through trial and error (91.60%) as well as freely available 
open educational resources (79.50%). Results are shown in Table 4. While trial and error along with accessing open 
educational resources (OERs) may lead to developing competencies associated with the design and development of 
e-learning products, more information is needed this approach to determine appropriate guidelines. 
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Table 4 

Resources leveraged to learn e-learning authoring tools 


Yes No 

1. Trial and error 91.60% (76) 8.40% (7) 

2. Open Educational Resources* (OERs) 79.5% (66) 29.50% (17) 

3. Friend 47.00% (39) 53.00% (44) 

4. Purchased resources** 4.80% (4) 95.20% (79) 

*OERs included videos, tutorials, and job aids. 

**Purchased Resources included videos, tutorials, job aids, and courses. 


Research question 4.1: How does the amount of prior work experience relate to learning methods selection? 

No significant correlations were found between the amount of prior work experience and the learning 
methods selection. This adds perspective to our findings, and shows that there may not be enough widely known 
resources to support individuals who are developing competencies associated with the design and development of e- 
learning. Furthermore, it might be worthwhile for the respective software companies to consider extending their 
marketing efforts to create a larger learning community around their e-leaming authoring tools. It could add to their 
positive branding by doing so for novice users. 

Table 5 


Correlation between experience and learning method selection 



Purchased 

OER 

Friend 

Trial and error 

Spearman’s Correlation Coefficient 

.117 

.136 

-.056 

.152 

Sig. 

.293 

.220 

.617 

.169 

N 

83 

83 

83 

83 


Note. Years of experience were reported on a scale of 1 - 5: 0-11 months (1), 1-3 years (2), 4-7 
years (3), 8-10 years (4), 10+ years (5). Purchase, OER, friend, and trial and error, resource 
selection was reported as no (0) or yes (1) by participants. 


Research question 4.2: How do competency levels relate to learning methods selection? 

Only two significant correlations were found between competency levels and learning methods selection. 
Developing competencies and OERs were significantly correlated for Captivate, r,(82) = .293, p = .007. Also, 
developing competencies and purchased formal training materials were significantly correlated for Lectora, r,(82) = 
.295, p = .007. Results are presented in Table 5. Again, this finding would suggest that desirable, quality-training 
materials are not widely available for those who are trying to develop tool competency. 
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Table 5 


Correlation between tool competency’ and resource selection 


Tool competencies 

Purchased 

OER 

Friend 

Trial and error 

Articulate 

Spearman’s Correlation Coefficient 

.014 

.16 

.069 

.072 

Sig. 

.897 

.148 

.534 

.516 

N 

83 

83 

83 

83 

Camtasia 

Spearman’s Correlation Coefficient 

0.009 

.09 

.074 

-0.013 

Sig. 

.939 

.42 

.508 

0.906 

N 

83 

83 

83 

83 

Captivate 

Spearman’s Correlation Coefficient 

0.001 

.293 

0.026 

.172 

Sig. 

.991 

.007* 

.817 

.12 

N 

83 

83 

83 

83 

Lectora 

Spearman’s Correlation Coefficient 

.295 

0.063 

.086 

.185 

Sig. 

.007* 

.57 

.441 

.094 

N 

83 

83 

83 

83 

PPT 

Spearman’s Correlation Coefficient 

0.167 

0.161 

.073 

-0.105 

Sig. 

.132 

.147 

.51 

.346 

N 

83 

83 

83 

83 


Note. Participants rated their tool competency on a scale of 1 - 5, from not competent to very 
competent. Purchased, OER, friend, and trial and error, resource selection was reported as no (0) 
or yes (1) by participants. 


Conclusion 

Several other technical fields (e.g., math, engineering, medicine, architecture) provide learners enrolled in 
degree programs the opportunity to gain formal instruction on project design as well as the tools needed to complete 
the project. There may be a growing need to package the traditional instruction provided on good ID practices with 
efficient and effective training on e-learning tool selection and use, for degree programs that are preparing 
instructional design professionals to gain entrance to positions and advance their careers. Additional research into 
the desired skills and competencies associated with available instructional design or training professional positions, 
hiring managers’ current selection practices for identifying successful candidates from applicant pools, and 
placement statistics of recent program graduates, would help clarify the actual demand and need for individuals with 
competencies associated with e-leaming authoring tools. 

Three themes emerged from the reported selection methods for authoring tools: tool availability, tool 
compatibility with project scope, and designer or developer comfort level with tools. Additional research should be 
considered to explore potential relationships between experience or competency and approaches to selecting e- 
learning tools. Also, we should find out if formal training on tool use and selection guidelines helps instructional 
designers and developers successfully advocate for more relevant tool selection based on the project needs with their 
clients. 

Those enrolled in degree programs tend to lack strong competencies in e-learning design and the use of 
authoring tools to create e-learning. Trial and error is most often leveraged to leam how to use e-learning authoring 
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tools, while purchased resources are least used. Most people also use open educational resources. Further 
investigation is needed to determine which OERs are most often used and whether or not a structured, formal 
training experience would support more efficient and effective development of e-learning design and authoring tool 
development competencies. 

Additional participant samples drawn from a large research university, different areas in the US, 
organizations that employ or support instructional designers, developers, or performance improvement 
professionals, and similar sampling pools from outside of the US would add value to this research project. The 
implications of this continued research may result in an ability to advocate for the funding to build and for the 
inclusion of formal training materials in degree programs for those who want to build e-learning tool competency. 
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Abstract 

Turkey’s Movement of Enhancing Opportunities and Improving Technology (FATIH) project is a 
nationwide program for integrating technology into instructional activities in classrooms. The project has 
involved approximately 750,000 teachers, and has a budget of eight billion dollars. This research literature 
review introduces the FATIH project in detail. The project has five main components: providing equipment and 
software substructure; providing educational and management of e-content; using effective information and 
communication technology (ICT) in teaching programs; administering in-service training of teachers; and 
providing conscious, reliable, manageable, and measurable ICT usage. The project started in November 2010, 
and is expected to be completed within five years from the start date. After four years, the project is not yet at 
the point of progress that was originally desired. Research findings show that the interactive board was the most 
commonly used technology during the project. With the project, teachers have saved time in course preparation; 
completed their own classes in a less time; been physically less tired; and had access to increased and diversified 
resources. However, teachers have complained about some classroom management issues and lack of electronic 
content. The teachers also need more effective professional development programs. 

Introduction 

Technological developments and new trends in educational-instructional applications require 
governments to initiate educational reforms. With this in mind, Turkey started supporting the “Movement of 
Enhancing Opportunities and Improving Technology” project. Its name in Turkish is “Firsatlari Artirma ve 
Teknolojiyi Iyile§tirme Hareketi” and its acronym is FATIH. The project’s official web page is at 
fatihprojesi.meb.gov.tr. The general aims of FATIH Project are (1) equating opportunities in education, (2) 
supporting the effective use of information and communication technology (ICT) tools such as interactive 
boards (IBs) and tablet PCs in the classrooms (Akgun, Yilmaz, & Seferoglu, 2011; Balci, Gokkaya, & Kar, 

2013; Banoglu, Madenoglu, Uysal, & Dede, 2014; BMI, 2014; Cuhadar, 2014; Duman, Rural Baykan, Koroglu, 
& Yilmaz, 2014). The project started in November, 2010, after approval from the Ministry of National 
Education (MoNE) and the Ministry of Transport, Maritime Affairs and Communications. The FATIH project 
was funded by the Ministry of Science; Industry and Technology; Ministry of Economy; Ministry of Finance; 
Ministry of Development; Undersecretaries of Treasury; and Scientific and Technological Research Council of 
Turkey (TUBITAK) (Akgul, 2012; Banoglu et al., 2014; Ekici & Yilmaz, 2013). The project was targeted to be 
completed within five years from the start date. During the first three years of the project, the following were 
delivered to the schools: 614,364 laptops and data video projectors, 38,688 multifunctional printers, 600,000 
IBs, and over 15,000,000 tablet PCs. At the same time the electronic material was developed and teacher 
training provided. In the remaining two years of the project, feedback was being obtained from teachers, 
students, and school managers through the use of observations and surveys (Akinci, Kurtoglu, & Seferoglu, 
2012; Akgun et al., 2011; Akgul, 2012; BMI, 2013a , 2014; Eryilmaz & Salman, 2014; Yildiz, Saritepeci, & 
Seferoglu, 2013). This extensive and ongoing project has the following five main components (Alkan, Bilici, 
Akdur, Temizhan, & Cicek, 2011; Balci et al., 2013; MEB, 2015b): 

• supplying equipment and software substructure; 

• providing educational e-content and management of e-content; 

• promoting effective usage of the ICT in teaching programs; 


1 This study was supported by The Scientific and Technological Research Council of Turkey (TUBITAK) 2219 
scholarship program. 
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• offering in-service training of teachers; and 

• promoting conscious, reliable, manageable, and measurable ICT usage. 

Providing Equipment and Software Substructure 

Each preschool, primary school, and secondary school was given a multifunctional printer and camera. 
Each classroom was equipped with an interactive board and wired internet connection. Each teacher and student 
was given a tablet PC (Eryilmaz & Salman, 2014; MEB, 2015d). It was expected that a large budget would be 
needed for the FATIH project due to hardware and software needs. However, there were different thoughts 
about the amount of budget needed. According to Akgun et al. (2011) and Uluyol (2013), the project would cost 
approximately 1.5 billion U.S. dollars. On the other hand, the prior Minister of National Education, Mr. Omer 
Dincer, announced that the total budget of FATIH Project was approximately 4 billion U.S. dollars (“Total cost 
of the FATIH Project”, 2014). When the reports from the Business Monitor International (BMI) (2013a, 2013b, 
2014) were analyzed, the government revised the budget to 7.5 billion dollars. Investments in FATIH project are 
given in Tablel and are based on MoNE budget presentations (Kiranli, Gungor, & Yildirim, 2014). 

Table 1. The investment in FATIH Project 


Years 

Budget 

(U.S. 

dollars) 

Multifunction 
al Printer and 
Digital 
Camera 

Distanc 

e 

Learni 

ng 

Center 

Number of 
School 
Interactive 
Board Setting 
Up 

Interactiv 
e Board 
Set Up 

Tablet 

PC 

Deliver 

y 

2012 

400 

Millions 

3,657 


52 

84,921 

13,500 

2013 

700 

Millions 


110 

3,362 


49,300 

2014 

(Planned 

) 

u 

Billions 

28,351 



347,367 

675,00 

0 

Total 

2,2 

Billions 

32,008 

110 

3,414 

432,288 

737,80 

0 


According to Ekici and Yilmaz (2013), a contract was signed with a local electronics firm for 84,921 
IBs in November 2011. The IBs required Windows and Pardus operating systems, as well as StarBoard and 
Libre Office software. Pardus is a national operating system and developed by TUBITAK. Tablet PCs have an 
Android system which is an open source operating system. The Minister of National Education, Mr. Nabi Avci, 
announced that 62,800 tablet PCs were delivered to teacher in October 2013 (“ What is the last situation about 
FATIH Project", 2013). 

Until the end of 2013; 62,800 tablet PCs, 84,921 IBs, 3,657 document cameras, and 3,657 
multifunctional printers were distributed. Completion of 3,362 schools local area network infrastructure setup 
was successfully accomplished. A contract was signed for 675,000 tablet PCs, 347,921 IBs, 13,645 A3 printers, 
and 28,351 A4 printers. In 2014, 550,000 tablet PCs for students and 125,000 tablet PCs for teachers were 
distributed (MEB, 2014). Since some bidding processes were cancelled in 2014 and 2015, enough IBs and other 
equipment have not yet been distributed. 

Providing Educational e-content and Management of e-content 

The FATIH project was developed to promote e-leaming so that students become actively engaged and 
meaningful learning would be accomplished. Another aim of the project was to provide equality of opportunities 
for all students. Perhaps the most important goal is the continuing development of electronic content for 
educational purposes. Both MoNE and TUBITAK have joint works on material development in accordance with 
the criteria (Ekici & Yilmaz, 2013). These electronic materials can be accessed either online or offline in a web- 
based environment (Alkan et al., 2011). The contents are serviced through the network developed by MoNE. 
Education Information Network (Egitim Bilisim Agi in Turkish and abbreviated as EBA; official web page is 
eba.gov.tr) is a social platfonn where the content is delivered. 

Approval of the materials has been conducted according to set criteria that were determined to be 
appropriate, reliable, and valid by the Innovations and Educational Technologies General Directorate (MEB, 
2015a). The EBA platfonn has five main components: an educational search engine, educational content, 
classroom materials, educational games, and a question-and-answer session for students (Alkan et al., 2011). By 
August 30, 2015, EBA website had over two million registered users, 6,234 news items, 56,432 visual materials, 
9,224 videos, 4,395 audio files, 1534 journals, 2,908 documents, and 1,898 e-books. 
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Providing Effective Usage of the ICT in Teaching Programs 

Updating the teacher guidebooks of preschool, primary school, and secondary school teachers; 
developing curriculum manuals; and providing teaching activities on the hardware; and then implementing 
effective use of electronic documentation have been conducted (Alkan et al., 2011; Ekici & Yilmaz, 2013). At 
the same, time some in-service training programs have been offered to teachers in order to encourage effective 
usage of the ICT equipment in the classrooms in the learning- teaching process. 

Offering In-service Training of the Teachers 

The in-service training of approximately 750,000 teachers is on schedule. The content of the training 
covers the use of hardware in the classrooms and the effective pedagogical use of electronic educational 
materials (Alkan et al., 2011; Balci et al., 2013; Banoglu et al., 2014; Duman et al., 2014; Eryilmaz & Salman, 
2014; MEB, 2015c). In 81 provinces, 1 10 distance learning centers were built for the distance education 
applications. Training videos are uploaded to the kursiyetnet.meb.gov.tr website (Uluyol, 2013). As of July 20, 
2014, this website was been visited by over eight million people. Trainer education programs have been 
supported by the Innovations and Educational Technologies General Directorate and the Teacher Training and 
Development General Directorate. In the first part of the project, almost all of the 1 10,000 teachers in 3,662 
secondary schools received the following training programs: 

FATUT Project Seminars (8 hours): After the completion of the hardware setup in each school, these 
infonnative seminars are given before basic training (MEB, 2015c). 

Preparatory Training (25 hours): This training is given to the teachers who have not take any courses 
related to the use of computer or who felt they had inadequate computer skills. This training provides 
instruction to the participants on basic computer skills, operating systems, Internet applications, image 
processing, word processing, presentations, and measurement-evaluation (Ekici & Yilmaz, 2013; 

Yildiz et al., 2013; MEB, 2015c). 

Training Program of Using Technology in Education (30 hours): This training program is for teachers 
who have basic skills using computers. Trainer education program is about 36 hours (MEB, 2015c). 

The training includes important topics such as: basic infonnation about FATIH project; set up and use 
of IT equipment; basic concepts of using technology in education; use of appropriate materials in 
teaching process, material development; revising materials; teaching with IBs; and efficiency and 
evaluation of the effectiveness of materials (Ekici & Yilmaz, 2013; Yildiz et al., 2013; MEB, 2015c). 

Pardus Training Program (25 hours): This program promotes and provides information on the use of 
Pardus, one of the IBs operating systems (MEB, 2015c). 

Training Program about Information Technology and Secure Use of Internet (10 hours): This program 
is intended for teachers to guide students in use of ICT and Internet. Seminars are organized by both 
MoNE and the Presidency of Telecommunication (MEB, 2015c). 

FATIH Project Technology and Leadership Forum Training Program: This distance learning training 
program has two main parts. After a 10-day compliance program, there is face-to-face training for three 
days (24 hours total). This program covers educational technologies in Turkey and in the world; 
educational technologies investments in Turkey and FATIH project; trends in learning sources; 
investigation of teaching-learning materials; social media; Internet technologies; Internet ethics; 
bullying at school and on the Internet; Web 2.0 tools; analysis of the leadership behaviors and 
standards; technology work plan and integrating into FATIH project; and presentation of IBs and tablet 
PCs (Ekici & Yilmaz, 2013). This training program is for school administrators (Yildiz et al., 2013). 

The names of the in-service training programs and the number of participants are given in Table 2 
(MEB, 2015c). 
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Table 2. In-service Training Programs in FATIH Project 


Name of the in-service 
training program 

Duratio 

n 

(hours) 

Type 

Number of 
Participants 

FATIH Project - Using 
Technology in 
Education 

36 

Global - Face to face - For trainers 

1,571 

(2011,2012,2013) 




FATIH Project - Using 
Technology in 
Education (2012,2013) 

30 

Local - Face to face - For teachers 

105,000 

FATIH Project 
Informative Seminars 
(2012,2013) 

8 

Local - Face to face - For teachers 

73,000 

Administrative 
Seminars (2012,2013) 

24 

Global - Face to face - For 
administrators 

3,100 

FATIH Project - Pardus 
Training Program 
(2013) 

25 

Global - Face to face - For trainers 

216 

Training Program 
about IT and Secure 
Use of Internet (2013) 

24 

Global - Face to face - For trainers 

400 

CISCO (2013) 

60 

Global 

143 

FATIH Project 
Network Infrastructure 
Seminars (2012,2013) 

24 

Global 

594 



Total number of participants 

184,024 


As described in the previous paragraph, a variety of in-service training programs were organized during 
the project. Yildiz et al. (2013) evaluated the in-service training activities within the scope of the FATIH project 
in reference to the International Society for Technology Education (ISTE) standards. The evaluation findings 
suggest that these training activities do not contribute to the teachers’ professional growth in information 
technology literacy, possibly because these in-service training activities were covered too superficially. 
Generally, the training programs are criticized in terms of content and methodology. It is believed that teachers 
cannot attain the desired level of technology utilization with only a few weeks of in-service training; therefore, 
the human infrastructure should be in place before the implementation of such a large project as FATIH 
(Banoglu et al., 2014; Cengiz, 2012). 

Providing Conscious, Reliable, Manageable and Measurable ICT Usage 

Future plans have been made to establish a controllable network infrastructure, the supply of centrally 
controlled internet access, an N-tier firewall, and a manual covering the effective use of ICT. In particular, this 
component aims at increasing of awareness of students and teachers about safe usage of ICT and increasing co- 
operation among teachers and parents to help students with these safety concerns (Alkan et al., 20 1 1 ; Banoglu et 
al., 2014; Ekici & Yilmaz, 2013). 

In compliance with the official document of MoNE, it is obligatory for teachers of the public schools to 
take the 10-hour course entitled, “The Conscious and Safe Usage of Internet and Information Technology.” This 
course covers the conscious and safe use of the ICT and Internet; the risks in the use of Internet; law, human, 
and technical dimensions of ICT and Internet; the health aspects of safe use of ICT and Internet; Internet 
addiction; electromagnetic effects; psychosocial aspects; ethical rules; and digital citizenship. 

The Impact of FATIH Project 

The pilot implementation process focused on the evaluation of FATIH project by soliciting teachers’ 
opinions (Kurt, Kuzu, Dursun, Gullupinar, & Gultekin, 2013). Focus group interviews have been carried out 
with 52 teachers from five pilot schools in four different provinces. As a result of the focus group interviews, it 
has been concluded that the IB was the most commonly used technology during the project. Teachers' attitudes 
and interest for the use of technology; their access to different resources and course content; and their 
technology competence and awareness in the use of IBs, has been increasing. Teachers stated that because of 
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project they have saved time in course preparation; completed their own classes in a less time; been physically 
less tired; and had access to increased and diversified resources. Teachers reported that their technology 
competence has increased. Teachers also reported increased teacher-to -teacher, teacher-to-student, and student- 
to-student interactions. However, teachers have complained about some classroom management issues. For 
example; they cannot make eye contact with the students while using the IBs. 

Based on feedback from participating teachers and students in 1 1 pilot schools in five different cities 
(Pamuk, Cakir, Yilmaz, Ergun, & Ayas, 2013), it has been found that there has been a general positive attitude 
towards IBs and an increase in their use. On the other hand, the use of tablet PCs has been low. In addition to 
technological problems, the lack of the electronic contents is an another important problem. One of the most 
important findings is the professional development needs of the teachers on the use of technological issues 
provided in the project. There has been an increase in interest and motivation of teachers towards the use of 
technology and an expectation of teachers that these technologies have been a positive contribution to the 
learning and teaching processes. After interviews with 180 9th grade students and 50 teachers (Eryilmaz & 
Salman, 2014), it was found that students’ interest towards technology-enhanced lessons has increased because 
of ICT tools. It was also determined that the use of e-content (video, animation, e-books, cartoons, educational 
games, etc.) in the teaching process may facilitate the learning process. 

Keser and Cetinkaya (2013) and Cetinkaya and Keser (2014) analyzed the opinions of different 
teachers (n = 77) and students (n = 409) who have used IBs and tablet PCs with active learning strategies in 
their classes during the 2012-2013 academic school year. The researchers found that the first problem the 
teachers and the students faced was the misuse of IBs and tablet PCs in the educational setting. In particular, 
students often used IBs for listening music or watching movies. At the same time, teachers stated that they 
encountered technical and pedagogical problems while using IBs. There are also issues with tablet PCs in terms 
of poor classroom interaction and lack of appropriate material on the tablet PCs. There are also constraints in 
terms of assessment of knowledge and skills. Ayvaci, Bakirci, and Basak (2014) grouped the problems in terms 
of administrators, teachers, and students based on the results of the survey they conducted. During the FATIH 
implementation process, administrators and teachers had on three main problems: lack of in-service training of 
teachers, lack of teachers’ technology literacy levels, and lack of infrastructure. There were four core student- 
related issues related to tablet PCs: lack of installed programs as MS Word and MS PowerPoint, difficulties of 
using a tablet PC, limited Internet to search, and operating problems of tablet PCs, lessons (motivation 
problems, distractions, and lack of participation to course activities), communication (lack of communication 
between student-student and student-teacher, reduction in class discussions, and inability to socialize), and 
health care (the deterioration of eye health and the skeleton, and low back pain). Some researches (Dursun, 
Kuzu, Kurt, Gullupinar, & Gultekin, 2013; Kalelioglu & Altin, 2013) especially focused on school 
administrators’ views for FATIH Project. These studies found that school managers had a positive opinion, but 
they had concerns since the project had barely has enough content and technical problems occurred. 

In conclusion, this project may contribute to forming a technology culture, to encouraging the 
widespread use of ICT, and to developing students’ meaningful learning. But there are some problems related to 
the progress of the project. Not enough of the necessary equipment as IBs, tablet PCs, and other technical 
devices has been delivered to schools, teachers, and students during first four years of the project. In addition, 
the teachers need more effective professional development trainings and e-course content offerings. University 
cooperation is a necessity for the success of training programs. Faculty members in the departments of 
education, computer education, and instructional design in Turkey must take more responsibility. More research 
is needed to evaluate FATIH project as it continues, and more project studies should be supported in order to 
provide additional learning activities and training programs for success implementation of the project. 
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Accelerated Engagement of African-American Males 
Through Social Media 

Charles Holloway 

Introduction 

Given the social and economic problems African-American males face in the United States, their 
experiences at colleges and universities have become a source of major concern. One of the challenges 
most universities face today is the recruitment and retention of students. The retention rates and graduation 
rates of white males are followed by white females, and then African-American females. As current 
research shows, African-American males are going to college, but their persistence rates to graduation are 
significantly lower than their matriculation rates (Allen, 1992; Harper, Carini, Bridges & Hayek, 2004; 
Strayhorn, 2008). 

The following are some of the issues or concerns related to African-American males and their 
attendance to and completion of college. In 2008, only 47 percent of African-American males graduated 
from high school on time compared to 78 percent of their white counterparts (Harper, 2012). African- 
American males are less academically prepared for college (Harper, 2012). In 2002, African-American 
males comprised only 4.3 percent of the males enrolled in higher education which was the same percentage 
in 1976 (Harper, 2012). Black male completion rates are the lowest among both sexes and all racial and 
ethnic groups in the United States (Harper, 2012). The six year graduation rate for African-American males 
attending public colleges and universities was 33 percent compared to 48.1 percent for other students 
(Harper, 2012). 

The post-baccalaureate outcomes for some African-Americans are also concerning. For example, 
even after obtaining a degree, African-American males still experience lower employment rates compared 
to their white counterparts, which negatively affects the income they are able to earn even with degrees 
(McDaniel, DiPrete, Buchmann, & Shwed, 2011). In the professional arena, African-American males earn 
73 percent of the income of white males, with the average college-educated African-American male 
earning less than the average white male with a high school diploma (Jenkins, 2006). 

African-American parents, like all parents, play a pivotal role in the socialization of children, 
helping them to understand norms, roles, statuses, and expectations of the larger society (Mutisya & Ross, 
2005). Parental socialization is important, but other socialization agents that must be understood are 
gender, race, and sexual orientation because they are crucial to identity formation for children. Racial 
socialization attempts to prepare African-American children for the realities that African American face in 
America (Mutisya & Ross, 2005). Parents teach, set moral standards, discipline, and provide cultural 
continuity within a community (Mutisya & Ross, 2005). 

In order for students to be successful in college, they need a support system that assists them with 
adapting to a new environment. Institutional support and academic achievement are both important for 
student success. For example, the positive impact of a supportive environment has assisted African- 
American students at historically black colleges and universities (HBCUs) (D’Augelli & Hershberger, 
1993). Even though African-American students score below their counterparts on undergraduate 
admissions tests, even after controlling for family income and parental level of education, those that attend 
HBCUs have lower high school GPAs, ACT scores, and SAT scores than those who attend predominantly 
white institutions (PWIs) (Kim & Conrad, 2006). Students that attend HBCUs are also more likely to come 
from families with lower financial means to assist students with their education (Kim & Conrad, 2006). 

At PWIs, African-American students’ experiences with the environment have been mixed. Some 
must deal with cultural adjustments, isolation from other African-American students, and problems of 
racism, which affects their persistence (D’Augelli & Hershberger, 1993). If PWIs are not geographically 
close to African-American students’ homes, students must make significant adjustments that relate to 
personal, family, and social dynamics (D’Augelli & Hershberger, 1993). Most African-Americans come 
from communities where they were in the numerical majority, and now they are the minority. Therefore, 
they also must deal with racism in new ways. Even if a student has not faced racism personally, they know 
of others who have experienced racism (D’Augelli & Hershberger, 1993). 
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Getting Students Engaged 


In the year 2020, the majority of the U.S. population will be people of color (Floyd, 2001). As a 
global society, there are roles and responsibilities that all have as they contribute to society in a meaningful 
way. By matriculating successfully through college, African-American males will be able to lead 
productive lives and contribute to society. However, the barriers that inhibit their progress must be 
addressed. 

One thing that we have noticed is that it is hard to get students to check email or to respond to 
emails, but students do not mind sending text messages, tweeting, or using Facebook or other social media. 
Through social media, we will establish a Twitter account to get African-American males engaged on 
campus. The research will provide them with tweets to remind them of certain events on campus, provide 
academic information as needed, and also will provide quotes or words of encouragement to assist with the 
barriers they face. Currently, we are engaging 13 African-American men on campus and have established 
an informal networking group (MEN - Men Empowerment Networking) to support them through this 
semester. A request will be sent to three individuals to follow the Twitter account and business cards will 
be hand out to other students to encourage them to follow the account. This will allow for me to 
intentionally interact with the students because most of them already have accounts. The most important 
part of using social media is it will accelerate getting information to students quickly and often. 

Next Progression 

Institutional data will track the retention of the students from 1 st to 2 nd year as well as their 
persistence to graduation. A comparison of the retention rate of our African-American males with white 
males with be analyzed. In addition, the success rates of the African-American males who follow the 
Twitter account with those who do not with be performed. The followers, retweets, and other information 
that should assist the young men will be evaluated as they obtain their college education. 
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Abstract 

Because prior beliefs have been found to affect how people argue and respond to controversial claims, this 
study examined how patterns in students’ responses posted in online debates were associated with students’ initial 
positions on given claims prior to debate. Students that held initial opposing positions to the claim exhibited the 
tendency to respond to challenges from the opposition with explanations and to respond to explanations from the 
opposition with challenges, but not so among students with initial supporting and neutral positions. These findings 
suggest that manipulating group composition based on prior beliefs and manipulating the direction and phrasing of 
the claim under debate may help to elicit more frequent exchange of opposing viewpoints and raise the level of 
critical discourse. 

Introduction 

Collaborative argumentation is an instructional activity for fostering critical discussions in both face-to- 
face and online environments. Argumentation involves the process of building arguments to support a position, 
considering and weighing evidence and counter-evidence, and testing out uncertainties to extract meaning, achieve 
understanding, and examine complex ill-structured problems and issues. This process not only plays a key role in 
increasing students’ understanding but also in improving group decision-making. Online discussion boards are being 
increasingly used to engage learners in dialogue in order to promote more in-depth discussions. However, studies 
show that the quality of online discussions is often shallow. 

Given that argumentation is a highly cognitive form of activity, there are particular fallacies and biases in 
students’ reasoning processes that can inhibit the quality of argumentation in online discussions and/or debates. For 
example, prior research (Tversky & Kahneman, 1974; Nickerson, 1998; Correia, 2011) has documented how 
people’s prior viewpoints can bias how they process and respond to claims on controversial issues. The more people 
are strongly opposed to a particular claim or issue, the more likely they are to challenge and/or dismiss the 
arguments presented by the opposition. In contrast, people who have less emotional connection or place lesser 
importance on the claim under debate are more likely to engage in balanced and more sustained discussion and 
argumentation to examine and test the veracity of a given claim. 

The purpose of this study was to examine how students’ initial positions on particular claims under debate 
respond to the arguments, challenges, explanations, and supporting evidence presented by students on the opposing 
position in online debates hosted in asynchronous threaded discussion boards. Given the prior research and findings, 
this study hypothesized that students that are initially opposed to the claim under debate will question/challenge the 
postings from the opposition at a higher than expected frequency, but no so much the case for students with initial 
supporting and neutral positions on the claim. As a result, the questions examined in this study were: 

1 . What patterns exist in the way students respond to postings from the opposing team when a student 

possesses an initial opposing position, supporting position, and neutral position on the claim under debate? 

2. What differences exist in response patterns between these three groups of students? 
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Method 


The participants were 16 graduate students in an online course on distance learning at a large southeastern 
university. Each student participated in four online debates hosted in Blackboard discussion forums. Students were 
divided into supporting and opposing teams to debate for or against a given claim. Students were required to insert a 
tag into the heading of each posting to identify the posting as a supporting or opposing argument (+ARG /-ARG), a 
challenge (+BUT/-BUT), explanation (+EXPL/-EXPL), and supporting or counter evidence (+EVID/-EVID) with + 
and - tags to identify team membership (Figure 1). At the end of each debate, students completed an online post- 
debate survey that asked students to report their initial position on the claim and how their position changed (if any) 
following the debate. Based on survey responses, each student was identified by initial position (o = opposing, s = 
supporting, n = neutral). 


r SUPPORTING ARGUMENTS 

El +ARG1 MediaOevelopWorldKnowledge 
H+EVID1 MediaDevelop World Knowledge 

+EXPL MediaC hangeLearningEnvironments 
B Re: -BUT ExpNotOutcome 
+EXPL ExpNotOutcome 
B — BUTMediaDevelopWorldKnowledge 
B +BUT WeMustMakeMediaDecisions 
B -BUT WeMustMakeMediaDecisions 

+EXPL WeMustMakeMediaDecisions 
-BUT necessary Technology 

B-+SUPP ComputerSimulationsDevelopWorldKnowledge 
+EXPL 

B +ARG2 LearningAspectsInfluencedByMediaCharacteristics 
B— BUT LearningAspectsInfluencedByMediaCharacteristics 
B +EVID LearningAspectsInfluencedByMediaCharacteristics 
+EXPL 

-BUT LearningAspectsInfluencedByMediaCharacteristics 
-BUT2 Learning Aspects Influenced By MediaCharacteristics 
+EVID for LearningAspects Are InfluencedByMedia 


12/9/07 2:27 AM 
12/9/07 2:53 AM 
12/5/07 3:58 PM 
12/5/07 3:58 PM 
12/5/07 3:58 PM 
12/5/07 3:57 PM 
12/9/07 2:30 AM 
12/5/07 3:58 PM 
12/5/07 3:58 PM 
12/5/07 3:58 PM 
12/5/07 3:58 PM 
12/9/07 2:30 AM 
12/21/07 2:38 PM 
12/9/07 3:01 AM 
12/9/07 2:34 AM 
12/9/07 2:34 AM 
12/21/07 2:38 PM 
12/5/07 3:58 PM 
12/9/07 2:34 AM 
12/9/07 2:33 AM 


Figure 1. Screenshot of debate in Blackboard discussion forum with labeled messages. 


The message tags were modified to identify type of posting and initial position (i.e., BUTo = challenge 
posted by student with initial opposing position, EXPLs = explanation/justification posted by student with initial 
supporting position. The Discussion Analysis Tool (Jeong, 2005) was used to tally the number of times each type of 
message posted by one particular group of students (opposing, supporting, neutral) was posted in reply to another 
given type of message (posted by students across all groups) to generate the frequency and transitional probability 
matrices below. In figure 2, the frequency matrix (under opposing position) shows that challenges (BUT) elicited a 
total of 8 explanations from students that possessed an initial position in opposition to the debated claim (EXPLo), 
or 19% of all responses posted in reply to the challenges. This was significantly higher than the expected frequency 
based on the z-score of +2.26 at p < .05. In contrast, only 5% and 2% of the responses to challenges (BUT) were 
from students with supporting and neutral positions, respectively - in which neither of these observed frequencies 
were significantly higher than expected frequencies at p < .05. Altogether, these response probabilities were 
graphically conveyed using transitional state diagrams. 
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Figure 2. Screenshot of frequency, transitional probability, and z-score matrices produced by the DAT software 







Note'. Thickness of arrow conveys strength of transitional probability; dark black arrows identify probabilities 
significantly greater than expected based on z-score tests (p < .01) performed in the DAT software; first and second 
numerical value displayed in nodes identify the number of times the given action was performed and the number of 
events that followed the given action; the size of the glow emanating from each node conveys the number of times 
the action was performed. 


Figure 3. Transitional state diagrams depicting the response probabilities produced by each group 
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Main Findings & Implications 

A comparison of the three state diagrams (Figure 3) revealed that students with initial opposing positions to 
the claims revealed the tendency to respond to challenges from the opposition with explanation (BUT ->F,XPL) and 
the tendency to challenge explanations from the opposition (EXPL->BUT). In contrast, this message-response 
“pattern” (based on the z-score tests) was not exhibited by students holding initial supporting and neutral positions. 
To determine if this observed difference in response pattern was statistically significant, the Y ule’s Q-test was used 
to compute a phi-coefficient ( 0 ) to measure the strength of association between group membership and particular 
pattern (zero indicating no association, .90 an above indicating extremely strong relationship, .70 to .89 indicating a 
strong relationship, .50 to 69 a moderate relationship, .30 to .49 a low relationship, and .30 and below a weak 
relationship). The 40% vs. 29% of responses to challenges with explanations produced by students with initial 
opposing vs. supporting positions, respectively, revealed a low association with initial position (0 = 0.333; SE = 
0.415). However, the 40% vs. 17% of responses to challenges with explanations produced by students with initial 
opposing vs. neutral positions, respectively, revealed a moderate association with initial position (0 = 0.538; SE = 
0.421). 

The state diagrams also reveal that students with initial opposing positions to the claims exhibited the 
tendency to challenge explanations from the opposition (EXPL->BUT) - a response pattern that was not exhibited 
by students holding initial supporting and neutral positions. Although the 67% vs. 100% of responses to 
explanations with challenges were produced by students with initial opposing vs. supporting positions, respectively, 
the Yule’s Q test could not be used to test the significance of this observed difference due to low cell frequency and 
because the cell frequencies in the number of responses to explanations (other than challenges) were zero from 
students holding initial supporting positions. The 67% vs. 17% of responses to explanations with challenges 
produced by students with initial opposing vs. neutral positions, respectively, revealed a weak association between 
the EXPL^BUT response pattern and initial position (0 = 0.333; SE = 0.566). 

In addition, students with initial neutral positions showed the tendency to reply to arguments posted by 
members on the same side with explanations to strengthen their team’s position. This pattern was not exhibited by 
the students holding initial opposing and supporting positions. The 35% vs. 33% of responses to arguments with 
explanations produced by students with neutral opposing vs. opposing positions, respectively, revealed a no 
association with initial position (0 = 0.04; SE = 0.246). The 35% vs. 26% of responses to arguments with 
explanations produced by students with neutral vs. supporting positions, respectively, revealed a weak association 
with initial position (0 = 0.205; SE = 0.241). 

Overall, the findings to a certain extent are consistent with prior research (Tversky & Kahneman, 1974; 
Nickerson, 1998; Correia, 2011) that find that people whom are strongly opposed to a particular claim are more 
likely to challenge the arguments and claims presented by the opposition. The findings in this study provide insights 
into precisely how prior beliefs can affect the quality of argumentation in online discussions/debates. These findings 
suggest that the depth of argumentation can be increased by controlling group composition based on students’ prior 
beliefs and perhaps by phrasing the debated claim in two ways (or inversely) so that it is presented from an 
oppositional standpoint for the teams on both sides of the debate. These steps may help to promote more responses 
that challenge presented arguments and more rebuttals to challenges with explanations/justifications. However, 
future studies will be necessary to test the effects of these two and other possible interventions using controlled 
experimental design, conducted across discussions with multiple cohorts/groups to increase the sample size and to 
control for idiosyncratic differences in the social dynamics within groups, and applying a more conservative p-value 
for identifying patterns in the message-response sequences produced by different interventions. 
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Abstract 

Computer-based scaffolding (CBS) has been regarded as one of the most effective methods to improve K- 
12 students’ learning performance in STEM education. CBS plays a pivotal role in improving higher order skills 
required for successful learning in the context of problem-centered instructional models. The purpose of this paper is 
to examine the effectiveness of computer-based scaffolding by synthesizing the results of individual studies related 
to this topic through Bayesian Meta-analysis. The results show that computer-based scaffolding has the largest 
positive impact on students’ cognitive learning outcomes when it is used in project-based learning for Engineering 
education with a fading function determined by fixed schedule. 

Introduction 

The Next Generation Science Standards promote the use of problem-centered instructional approaches, 
which require students to construct knowledge in order to generate solutions to ill-structured authentic problems 
(Achieve, 2013). Central to student success in such approaches is scaffolding — dynamic support that helps students 
meaningfully participate in and gain skill at tasks that are beyond their unassisted capabilities (Belland, 2014; 
Hmelo-Silver, Duncan, & Chinn, 2007). Computer-based scaffolding, which appears to have overcome the 
limitations of one-on-one scaffolding from teachers (e.g. inadequate and conventional supports) (Belland, 2014), can 
provide individual and customized support through the continuous and dynamic diagnosis of students’ learning 
processes (Wood, 2001). 

In educational research, the experimental conditions (e.g. learning environments, learning contexts, student 
characteristics, and curriculums) can vary across the studies making it difficult to generalize the effectiveness of 
computer-based scaffolding. In this sense, meta-analysis to synthesize the results of studies of a similar research 
topic through the statistical techniques has been conducted (Copper & Hedges, 1994; Wilson & Lipsey, 2001). 

This paper aims to determine the most effective scaffolding strategies in K-Adult STEM education settings 
through meta-analysis techniques using the Bayesian approach, which overcomes several issues (i.e. the skewed 
distribution, the small size effects, and the heterogeneity) of traditional meta-analysis. 

Computer-based scaffolding 

Scaffolding, which is defined as dynamic supports such as questioning, expert modeling, and feedback, has 
helped students solve ill-structured and authentic tasks otherwise beyond their current capabilities (Belland, 2014; 
Wood, Bruner, & Ross, 1976). Recently, many studies have demonstrated that computer-based scaffolding, 
including intelligent tutoring systems and pedagogic agents, can be effective in enhancing students’ higher order 
thinking skills and content knowledge in STEM education (Belland, Glazewski, & Richardson, 2011; Hmelo-Silver 
& Day, 1999; Kramarski & Gutman, 2006; Leemkuil & de Jong, 2012; Li, 2001; Schrader & Bastiaens, 2012). 
According to the results of the meta-analysis synthesizing 7 studies, computer-based scaffolding has a positive 
impact (g=.53) on students’ learning in STEM education (Belland, Walker, Olsen, & Leary, 2015). A more 
extensive meta-analysis also revealed the large effects (g=.46) of computer-based scaffolding on improving 
students’ knowledge integration as well as higher-order thinking skills in the context of problem-centered 
instructional models for STEM education (Belland, Walker, Kim, & Lefler, under review). 
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Why Bayesian Meta-analysis? 


Traditional meta-analysis (TMA) is a statistical method that combines results from multiple individual 
studies that have the same or similar research questions (Hedges & Olkin, 1985). TMAs are often seen as thorough 
and accurate in that they can correct for sampling errors and measurement errors between studies. However, TMA 
still has unsolved issues related to (a) skewed distribution caused by publication bias, (b) effects from small number 
of outcomes, and (c) heterogeneity across studies (Smith, Spiegelhalter, & Thomas, 2007). 

Skewed distribution. One issue mentioned most often in relation to meta-analysis is publication bias. Publication 
bias means the tendency to publish positive results rather than negative or inconclusive results, resulting in an 
inaccurate representation of the population. According to a study investigating the publication bias of several recent 
meta-analyses (Kicinski, 2013), 42 out of 49 meta-analyses contain risk of publication bias. Due to publication bias, 
the distribution of effect sizes is always negatively skewed, which means that the mass of the distribution is 
concentrated on the right of the figure. This may be the cause of positively over-weighted effect-sizes in TMA. 
Small N effects. In TMA, effect sizes are estimated based on the observations of previous individual studies. In 
order for meta-analyses to be accurate these observations should be standardized, however errors can occur because 
some moderators are represented more widely than others across studies. If the number of outcomes for a certain 
moderator is relatively small compared to others, this moderator tends to report larger effect sizes than those with 
more outcomes. This is due to wider confidence intervals. Therefore, some scholars claim that if a data set includes 
a moderator with a small number of outcomes, this moderator should be excluded from the data set in order to 
prevent a biased effect size (Bailar, 1997; Greco, Zangrillo, Biondi-Zoccai, & Landoni, 2013; Hedge, 1986). 
However, in the field of education it is often difficult to find enough studies to provide a comparable representation 
of all variations of each moderator. 

Heterogeneity. In meta-analysis in education, the random effects model (REM) typically has been utilized to avoid 
the assumption of homogeneity (Higgins & Thompson, 2002). Heterogeneity among studies can result in wide 
confidence intervals or extreme values compared to the fixed effects model. Therefore, TMAs reported the / a value 
to show how much the included studies show heterogeneity. However, according to a recently published article by 
Borenstein, Hedges, Higgins, & Rothstein (2009), is not an absolute measure of heterogeneity, but the proportion 
of the observed variance reflecting variance in true effects. This means that it might be difficult to figure out 
whether the heterogeneity across the studies exists in reality as well as in statistics. 

Bayesian approach. An alternative to TMA, which addresses the above issues, is the Bayesian approach, which 
assumes that all parameters come from a superpopulation with its own parameters (Hartung, Knapp, & Sinha, 2008; 
Higgins, Thompson, & Spiegelhalert, 2009). This approach relies on (a) generating a prior distribution (.P(0)) 
utilizing data from pre-collected studies that should not be included in the Bayesian meta-analysis, (b) estimating the 
likelihood that the prior distribution is valid based on observed data (/■ > (0|data)), and (c) generating a posterior 
distribution which can be calculated by the Bayesian law of probability (/ > (data|0)). Therefore, this approach can 
provide a more accurate estimate of a treatment effect by adding another component of variability — the prior 
distribution (Schmid & Mengersen, 2013). The benefits of the Bayesian approach to meta-analysis are as follows. 
First, as when using the Gibbs sampling by probabilistic inference, the sample sizes can be huge, resulting in solving 
the issues of skewed distribution and small number effects in TMA. Second, one can fix the t 1 value, which means 
the variance between the true effect sizes (the variance between the studies) as the prior distribution, and in BMA 
the studies included in a certain moderator can have their own parameters. In this sense, BMA does not consider 
heterogeneity across the studies as controlling the variance between studies. 
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Research Questions 


The purpose of this paper is to investigate the possibility of more accurate and reasonable effect sizes of computer- 
based scaffolding through Bayesian Meta-analysis which resolves some issues of the existing data-set. In this sense, 
the research questions are as follows. 

1 . How skewed is the distribution of sub-categories within moderators from the existing literature related to 
computer-based scaffolding? 

2. To what extent does computer-based scaffolding affect students’ cognitive outcomes according to context 
of use, scaffolding intervention, scaffolding customization, education level, assessment level, and 
discipline? 


Method 


Searching process & criteria 

Employing appropriate search terms (scaffolding, computer, intelligent tutoring systems, and cognitive 
tutor), we searched journal articles, dissertations, and conference proceedings in the databases of Education Source, 
PsychINFO, Digital Dissertation, CiteSeer, Proquest, ERIC, PubMed, Academic Search Premier, and Google 
Scholar. 144 studies (333 outcomes) were selected to conduct Bayesian Meta-analysis as the basic information for 
likelihood. Studies done in the context of providing scaffolding in problem-centered instructional models, which 
were published between Jan, 1993 and Dec, 2014, were required to meet the following criteria: a) had a control 
group, b) had sufficient data to calculate the effect sizes, and c) measured cognitive learning outcomes. 

Scaffolding moderators for Meta-analysis 

Scaffolding types. Conceptual scaffolding guides students to construct and problematize tasks through hints and 
prompts (Hannafin, Land, & Oliver, 1999; Reiser, 2004). Metacognitive scaffolding encourages students’ 
consideration of multiple problem-solving paths through reflecting on their understanding and learning processes 
(Hannafin et al., 1999). Strategic scaffolding suggests steps or strategies for how students can utilize evidence to 
solve problems (Hannafin et al., 1999). Through motivational scaffolding students’ interest and self-confidence 
toward learning is enhanced (Tuckman & Schouwenburg, 2004). 

Scaffold customization (scaffolding change and schedule). Scaffolding customization means that scaffolding can 
be faded and added according to students’ current abilities and needs. When students can accomplish the ill- 
structured tasks without any scaffolding, the continuous provision of scaffolding can be obstructive to students’ self- 
directed learning and research. In this case, scaffolding should be faded. On the other hand, the nature of scaffolding 
should be changed or the amount of scaffolding should be added if students continue to have difficulty in 
progressing their learning even though scaffolding is being provided. The fading or adding of scaffolding can be 
determined by fixed time intervals, self-selected, or performance-adapted. 

Education level. Target populations in this paper cover a wide range of levels: primary (kindergarten-5 th grade), 
middle level (6 th - 8 th grade), Secondary (9 th - 12 th ), college, graduate, and adult. 

Context of use. Scaffolding can be used in the context of problem-based learning, project based-learning, case- 
based learning, modeling/visualization, design-based learning, inquiry-based learning, and problem-solving. 
Assessment level. This moderator represents which area students will be expected to enhance their learning 
performance through computer-based scaffolding. Concept level checks students’ concept knowledge and ideas. 
Principle level assesses student’ ability to form a relation between several concepts or knowledge. At the level of 
application, students should utilize conceptual and principle level knowledge to solve an authentic problem. 
Discipline. The purpose of this paper is to explore the effectiveness of computer-based scaffolding in STEM. 
Therefore, disciplines such as Science, Technology, Engineering, and mathematics are included as a moderator. 
Table 1 shows the sub-categories of moderators in this study and the number of outcomes from the raw data. 
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Table 1. The sub-categories of moderators from the raw data 


Education Level 

Assessment Level 

Discipline 

Scaffolding Intervention 

• Primary (n=28) 

• Middle (n=108) 

• Secondary (n=53) 

• College (n=132) 

• Graduate (n=ll) 

• Adult (n=l) 

• Concept (n= 125) 

• Principle (n=167) 

• Application (n=41) 

• Science (n=208) 

• Technology (n=51) 

• Engineering (n=30) 

• Mathematics (n=44) 

• Conceptual (n=227) 

• Metacognitive (n=28) 

• Strategic (n=75) 

• Motivational (n=3) 

Scaffolding Change 

Scaffolding Schedule 

Context of Use 

• None (n=216) 

• Fading (n=12) 

• Adding (n=62) 

• Fading & Adding 
(n=43) 

• None (n=216) 

• Performance-adapted 
(n=63) 

• Fixed (n=13) 

• Self-selected (n=41) 

• Project-based learning 
(n=5) 

• Problem-based learning 
(n=38) 

• Design-based learning 
(n=4) 

• Case-based learning 
(n=15) 

• Inquiry-based learning 
(n=69) 

• Problem-solving (n=38) 

• Modeling/visualization 
(n=42) 


Data Analysis 

Prior distributions can be based on subjective opinion, which leads to researchers’ bias (Egger, Smith, & Phillips, 
1997). Therefore, this study will use the normal prior distribution among non-informative prior-distribution 
approaches, not including specific information about the prior distribution (Carlin & Chib, 1995; Sutton & Abrams, 
2001). Gibbs sampling using the Markov chain Monte Carlo algorithm will be used to generate samples from a 
probability distribution of the Bayesian model (Dellaportas, Forster, & Ntzoufras, 2002). Integrating Markov chains 
can replace the unstable initial values of random variables with more accurate values through repetitive linear steps, 
in which the next state (i.e. value of variable) can be influenced by the current one, not by the preceding one (Neal, 
2000). For this process, 12,500 random numbers for estimation of posterior distribution will be generated and 2,500 
random numbers will be deleted to eliminate initial values that are randomly given. Therefore, the final sample size 
was 10,000. We utilized STATA 14 to conduct Bayesian Meta-analysis. 

Results 

Bayesian Meta-analysis corrected two important things: 1) the modification of effect size caused by a small number 
of studies and 2) the correction of a skewed distribution (possibly caused by publication bias) by generating big 
sample sizes. The overall effect sizes of computer-based scaffolding from 10,000 outcomes simulated by 333 
outcomes from the raw data was 0.53. The Bayesian Meta-analysis model assumes that each moderator has its own 
parameter compared to TMA in which the effect size of each moderator can be estimated under one parameter 
representing the effects of treatment. Therefore, the overall effect size from 10, 000 samples and the overall effect 
sizes from the sum of each moderator can be different. 

The skewed distribution 

When investigating the distribution of each moderator based on the raw data (n=333), we found that the distributions 
of some sub-categories within moderators were skewed. This skewed distribution can affect the over-estimated 
effect-sizes in traditional meta-analysis, and this result can explain why Bayesian meta-analysis, which can fix the 
skewed distribution by assuming the normal distribution of each parameter, can get more accurate effect-sizes of 
computer-based scaffolding. Table 2 shows sub-categories within moderators having the skewed distribution based 
on the result of the Doornik-Flansen test. 
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Table 2. The skewed distribution 


Subcategories-moderators 


Probability 

Adding support- Scaffolding Change 

12.421 

<.01 

Case-based learning - Context of Use 

7.229 

<.05 

Modeling/Visualization - Context of Use 

6.024 

<.05 

Problem Solving - Context of Use 

6.729 

<.05 

Strategic scaffolding - Scaffolding Intervention 

21.883 

<.01 

Principle - Assessment level 

6.818 

<.05 

Mathematics - Discipline 

6.065 

<.05 


The results of each moderator 

As can be seen in Table 3, Hedges’ g estimates of all moderators were significantly larger than 0 (p<.05). 


Table 3. The effect sizes of each moderator (sample size — 10,000) 


Moderators 

Hedges’ g 

Std. Dev. 


Equal-tailed 
[95% Cred. Interval] 

SCAFFOLDING CHANGE 

None 

.42 

.05 

.30 

.51 

Fading 

.51 

.11 

.33 

.73 

Adding 

.41 

.08 

.23 

.56 

Fading & adding 

.49 

.09 

.30 

.65 

SCAFFOLDING SCHEDULE 

None 

.50 

.05 

.39 

.59 

Performance Adapted 

.57 

.07 

.44 

.71 

Fixed 

.60 

.12 

.38 

.78 

Self-selected 

.53 

.07 

.39 

.66 

CONTEXT OF USE 

Case Based Learning 

.53 

.01 

.51 

.56 

Inquiry Based Learning 

.29 

.01 

.28 

.31 

Designed Based Learning 

.63 

.04 

.56 

.71 

Modeling/Visualization 

.30 

.05 

.20 

.40 

Problem Based Learning 

.33 

.04 

.26 

.41 

Problem Solving 

.27 

.02 

.23 

.31 

Project Based Learning 

.94 

.03 

.88 

1.00 

EDUCATION LEVEL 

Adult 

.16 

.03 

.11 

.21 

College/Vocational/Technical 

.46 

.05 

.38 

.54 

Graduate/Professional 

.55 

.10 

.35 

.69 

Middle Level 

.24 

.01 

.21 

.27 

Primary 

.50 

.03 

.43 

.57 

Secondary 

.28 

.07 

.14 

.43 

SCAFFOLDING INTERVENTION 

Conceptual Scaffolds 

.44 

.05 

.35 

.52 

Metacognitive Scaffolds 

.41 

.08 

.24 

.57 

Motivational Scaffolds 

.39 

.23 

.02 

.87 

Strategic Scaffolds 

.44 

.07 

.31 

.58 

ASSESSMENT LEVEL 

Concept 

.48 

.15 

.22 

.75 

Principles 

.54 

.14 

.28 

.79 

Application 

.50 

.15 

.23 

.77 

DISCIPLINE 

Engineering 

.53 

.10 

.35 

.74 
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Mathematics 

.50 

.12 

.26 

.74 

Science 

.34 

.06 

.22 

.45 

Technology 

.47 

.07 

.35 

.61 


Education level. The effect size of graduate level (g=. 55) is significantly higher (p<. 05) than those of secondary 
(g=.28), middle (g=.24), and adult level (g=,16). The effect size (g=.50) estimated by primary level was also higher 
(/;<.() 1 ) than adult, middle, and secondary level. In addition, the difference between college level and others (i.e. 
secondary, middle, and adult level) was significant (p<.05). 

Assessment level. The effect size of principles level (g=.54) was higher than concept (g=.48) and application levels 
(g=.50), but this difference was not statistically significant (/?>.05) 

Discipline. The effect size of computer-based scaffolding utilized in Engineering (g= .53) was higher than in 
Mathematics (g=.50), Technology (g=.47), and Science (g=.34), but only the difference between Engineering and 
Science was significant (z= 2.00, /K.05) 

Scaffolding intervention. Conceptual (g= .44) and Strategic scaffolding (g=.44) can have more impact on students’ 
cognitive outcomes than metacognitive (g=.41) and motivational scaffolding (g=,39). However, there were no 
statistically significant difference between those scaffolding interventions (/?>.05). 

Scaffolding change. Scaffolding including the fading function (i.e. fading (g=,51) and fading/adding (g=,49)) 
showed higher effects than scaffolding with no fading (g=.42) and adding (g=,41). The difference between 
scaffolding changes was not significant (/?>.05). 

Scaffolding schedule. In the case of scaffolding schedule, all sub-categories of scaffolding schedule (i.e. none 
(g=.50), performance-adapted (g=.57), fixed (g=.60), and self-selected (g=.53)) have a huge effect size and their 
effect sizes were significantly greater than zero (p<.()5). However, the difference between the effect-sizes of sub- 
categories in scaffolding schedule was not statistically significant (p>. 05). 

Context of use. The effect sizes of computer-based scaffolding have a wide range according to which problem- 
centered instructional models are being used. The effect sizes estimated in Case-based learning (g=,53). Project- 
based learning (g=.94), and Design-based learning (g=,63) were higher than Inquiry-based learning (g=,29), 
Problem-based learning (g=,33), Problem-solving (g=,27), and Modeling/Visualization (g=,30). The difference 
between Project-based learning and other models was statistically significant (/K.05). In addition. Design-based 
learning and Case-based learning also showed a statistically significant difference between effect sizes compared to 
the other models which have lower effect-sizes (p<.05). 

Conclusion 

The skewed distribution among empirical studies of intervention of computer-based scaffolding, context of 
use, assessment level, scaffolding change, and discipline makes it difficult to determine the true effect of computer- 
based scaffolding. Moreover, some studies or some moderators can be excluded due to a lack of numbers although 
studies satisfied all criteria for meta-analysis. However, Bayesian meta-analysis can even include a small number of 
studies while providing more accurate estimates of effect sizes. The use of Bayesian meta-analysis should contribute 
to an enhanced understanding of the effect of computer-based scaffolding on cognitive outcomes. The results of 
Bayesian Meta-analysis in this paper showed similar results to two previous meta-analyses conducted by Belland et 
al. (2015; Under Review). The results indicate that computer-based scaffolding has a significant impact on students’ 
cognitive outcomes related to STEM education in problem-centered instructional models. Especially large effects on 
students’ cognitive outcomes are seen when computer-based scaffolding with a fading function based on fixed 
timing can be used in the context of project-based learning for Engineering. Based on this result, if specifically 
investigating how computer-based scaffolding has been designed and developed in the above mentioned contexts, 
we can suggest the most effective scaffolding strategies for STEM education using Bayesian meta-analysis. 
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Introduction 

Change occurs over time in individuals by restructuring their perceptions of the world (Burnes, 2004; 
Prochaska & Veliver, 1997). Change theories related to individual development of self-efficacy is focused on 
confidence in his/her ability to take action and persist in that action (Kritsonis, 2004). As noted by Bertalanffy, 
humans have open systems that arrive at a final state through alterations and reactions to errors encountered in their 
surrounding environment (Richey, Klein, & Tracey, 2011). Today, the Prochaska, DiClemente, and Norcross 
Transtheoretical Model (TTM) stands as a well-known change theory that prepares and maintains desired behavioral 
changes in the face of social situations that trigger relapse (1992). By contrast, process models of change are 
sequenced, anticipated events, or essential steps, of implementation of identified tasks. Berger and Luckmann’s 
(1966) re-socialization through alternation (RSA) is a well-known socially constructed process model. According to 
Berger and Luckmann, alternation refers to complete re-socialization of structural identity. Through alignment of 
features found in both RSA and TTM, a modernized model to assist patients in their development of asthmatic self- 
identification and, ultimately, can promote changes in their health behavior. 

In 2013, the Centers for Disease Control and Prevention (CDCP) reported asthma prevalence to be (in 
thousands) 22,648, with a child population of 6,109 (age < 18) and adult population of 16,540 (age > 18). The 
number of asthma persons reported to having one or more attacks was (in thousands) 3,524 (57.9%) for children and 
7,770 for adults (47%) (CDCP, 2013). The CDCP report (2013) also identified asthma as being the primary 
diagnosis in 2010 and 2011 for 1.8 million emergency visits to the emergency departments, 14.2 million physician 
office visits and 1 .3 million hospital outpatient department visits. Additionally, and a more troubling statistic is the 
national asthma mortality rates are 218 for children and 3,412 for adults (CDCP, 2013). Work-related asthma 
affected 1.4 million adults according to an Asthma Call-Back Survey (ACBS) in the US from 2006 to 2009 
(Knoeller, Mazurek, & Moorman, 2012). Their 2012 study identified a higher work-related asthma prevalence based 
on age, race, and ethnicity: 12.7% for persons aged 45-64, 12.5% for blacks, and 11.8% for persons of other race. 

Because of the rise of asthma prevalence in the US, it is not enough to assess and prescribe medications for 
asthma. Additionally, current patient education practices do not appear to be sufficient to bring about change for 
these patients. Instead, asthmatic patient self-identification needs to take into account sociodemographic factors in 
order for patients to acquire new knowledge and personal identity in order to implement a change in their health 
behaviors. 


Education Programs for Patients with Asthma 

Asthma educational programs are in two general categories: environmental education and self-management 
education (Nurmagambetov et ah, 2011). Environmental education is generally segmented into themes of school, 
work, community, and family. Self-management education is based on themes of self-monitoring, empowerment, 
social-ecology, and self-efficacy (Bravata et ah, 2007). 

Children ages 5 to 17 accounts for an annual loss of more than 10.5 million school days per year (United 
States Environmental Protection Agency, 2013). Classmates and friends can encourage good asthma practices or 
become a barrier (Clark & Valerio, 2003). School related benefits of maintaining better grades, participating in 
recreational activities, and improved social adjustment to classmates and teachers are exhibited through learning 
sessions focused on: how to help the child take medicine (child and educator received written-action plan with 
inhaler directions), how to determine activity levels for planned exercise and organized sports, adjustment of 
breathing, relaxation, and medication adjustments based of peak expiratory flow meter results (Clark, Feldman, 
Evans, Wasilewski, & Levison, 1984; National Asthma Education and Prevention Program, 2007). School based 
curriculums, such as the American Lung Association’s Open Airways for Schools are aimed at taking steps to 
prevent symptoms, recognizing symptoms when they first occur, carrying out appropriate management steps, 
discussing and solving asthma-related problems with parents, medical professionals, teachers and friends, and 
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feeling more confident about taking care of asthma on a daily basis (American Lung Association, 2015). Peer 
education in which older students were trained to deliver education to younger students improved quality of life in 
participating students and demonstrated instructional strategies of gaming, reciprocal teaching, and reflection (Shah 
et al., 2001). School aged children with asthma who have greater feelings of self-competence know more about their 
disease, demonstrate active coping styles, and are able to transfer to other social and behavioral dimensions (Clark & 
Valerio, 2003; Mitchell & Murdock, 2002). 

Work-exacerbated and occupational asthma is caused or made worse by exposures to substances in the 
workplace such as chemicals, dust, mold, animals, and plants. Exposure can be via skin contact and inhalation, 
which can start at work or within several hours after leaving work with no clear pattern. Workers previously 
diagnosed with asthma may find their condition worsening due to recurrent exposures, but is highly situational. 
Beyond non-educational interventions through Occupational Safety and Health Administration (OSHA) safety laws 
and data-sheets in the workplace, educational programs and short-sessions for work-related asthma (WRA) are 
mostly served by clinics, community-based organizations, and hospitals. Adult-oriented asthma programs, as 
exampled by the Adult Clinic at National Jewish Health, does incorporate occupational therapies to asthma and over 
7-10 days implements individualized plans reflecting need for physical assessment, respiratory therapy, medications, 
nutritional support, and education (National Jewish Health, 2015). The traditional paper-based instructional delivery 
for adults with WRA focuses on: defining asthma, identifying asthma symptoms, using an asthma checklist to 
monitor symptoms, and recognizing allergens and irritants. Of concern with this delivery method is health literacy, a 
known factor influencing health and correlated with household income, educational attainment, and social support 
(Rosas-Salazar, Apter, Canino, & Celedon, 2012). For adults, mitigating exacerbations to WRA is closely tied to 
educational level, income, and perception (Janson, Earnest, Wong, & Blanc, 2008). Work-related asthma and 
subsequent paper-based instructional delivery can lack the oral communications strategies necessary for improving 
adherence behaviors; examples of communications strategies are ordering information from most to least important, 
speaking slowly and using simple words, and presenting only essential information (Rosas-Salazar, Apter, Canino, 

& Celedon, 2012). 

Social ecology subsumes individual self-identification, relationships, communities, and societal-cultural 
norms. Adolescent development changes are mainly physical and psychosocial, with recommended developmentally 
appropriate education programs that are respectful of growing autonomy, learner-focused, and generate self- 
regulatory skills (Bruzzese et al., 2004). In order to break through barriers, AE-C led educational interventions for 
adolescents have focused on extending appointment times (90 minutes) and incorporate families to the sessions for 
reviewing critical information, reviewing guidelines, and counteracting false messages and beliefs the family may 
hold (McDaniel et al., 2014). In addition, barriers to asthma education commonly addressed to parents and 
caregivers focused on remembering daily medication, concerns about possible medication side effects, health care 
provider hours, and understanding medication instructions or techniques (McDaniel et al., 2014). A more responsive 
chronic-service model incorporating multiple caregivers is suggested to reinforce and remind children and 
adolescents of medication maintenance, with informal follow-up outreaches being appreciated by patients and 
family members (McDaniel et al., 2014). This stands in stark contrast to the dogmatic weekly and monthly check-up 
appointments in which education may or may not be an emphasis. 

Using culturally-normed educational approaches can alter attitudes, beliefs, and physical management of 
the disease leading to improved health outcomes (National Asthma Education and Prevention Program, 2007). 
Understanding ethno-cultural practices can influence asthma care, with open-ended questions being a reliable 
instructional strategy to elicit informative responses. Strategies that implement harmless or beneficial remedies 
jointly with medication plans are suggested, as harmless traditional remedies should be left alone and those harmful 
immediately addressed and discontinued. The dominance of prescribed Western medicine in the United States do 
not account for belief traditions of most racial and ethnic minorities. The dismissal of racial and ethnic minority 
beliefs is most commonly reflected through language deficiencies between the clinician and patient and family 
(Manson, 1988). Not accounting for language barriers and cultural norms are a root source for patients omitting 
medication(s), no-showing office appointments, and frequenting the emergency room (Manson, 1988). 

Singular or joint instructional strategies of feedback, reflection, metacognition, and goal-setting in existing 
asthma educational programs are beneficial in the short-term (< 1 year), but lacks long-term (> 1 year) maintenance 
identification to lung disease, causes, and impact on routine of daily living (Borrelli, Riekert, Weinstein, & Rathier, 
2007; Campbell et al., 2006; Riekert, Borrelli, Bilderback, & Rand, 2011). To promote long-term asthma 
management the Expert Panel Review 3rd edition (EPR3) guidelines commissioned by the National Asthma 
Education and Prevention Program Coordinating Committee (NAEPP) identified key points for therapy: reducing 
impairment and risk through a step-wise approach to pharmacologic therapy, monitoring and follow-up, therapeutic 
strategies through education, implementation of a written action plan, and referrals to asthma specialists, such as 


118 



Pulmonologists and Certified Asthma Educators (NAEPP, 2007). We posit that constructing a systems-driven 
change model in a socially constructive framework can adapt to and transform asthmatic self-efficacy in the modern 
educational and skill-building environment. The purpose of integrating a change model with a process model is to 
offer a coherent systems-driven change model for asthmatics struggling with adherence to medications and 
avoidance of allergens and irritants. The two models integrated are the Transtheoretical Model (TTM) and re- 
socialization through alternation (RSA). 

Transtheoretical Model and Re-socialization through Alternation 
Transtheoretical Model 

In the TTM, behavioral change is a chained process occurring over time by the five stages of 
precontemplation, contemplation, preparation, action, and maintenance. Each stage requires a set of tasks to be 
accomplished before moving to the next, following a more spiral than linear pattern in how people change through 
overt and covert activities while modifying behavioral problems (Norcross, Krebs, & Prochaska, 2011). 

Experiential, cognitive, and psychoanalytic influences are identified with the precontemplation and contemplation 
stages; likewise, existential and behavioral influences are identified with the action and maintenance stages (2011). 

In the precontemplation stage, behavioral change is intentionally at a standstill. Although families, friends, 
and employees are aware of problems, individuals at this stage are unaware and then due to pressure or coercion by 
loved ones or employers, seek out assistance in facilitating change (Prochaska, Norcross, & DiClemente, 2013). 

During the second, or contemplation stage, individuals are cognizant of an existing problem and begins 
thinking about overcoming it. Despite mental urge to action, individuals remains inactive to make the change and 
struggles with positive self-evaluations and a loss in overcoming drive of inaction (Prochaska et al., 2013). Problem 
consideration and resolutions are keys elements of the contemplation stage. 

The third stage, preparation, begins with “baby steps” in reporting small behavioral 
changes with more actions in upcoming months (Norcross, Krebs, & Prochaska, 2011). 

Contemplation and action highlight this stage, with behavioral changes emerging through self- 
preparation and social empowerment. 

The action stage is signaled by patients beginning to modify behavior, experience, and environment to 
overcome problems. However, commitment to time and energy in modifying overt behavior may only last 1 day to 6 
months (Norcross et ah, 2011). Yet, at this stage, there is discernible recognition of behavioral changes. For 
example, individuals to make a choice and commitment to change while demonstrating appropriate decision-making 
(Prochaska et ah, 2013). 

The final stage is maintenance, which is described as the efforts to prevent relapse and consolidate gains 
that were realized during the previous stage. Individuals that are free of the problem from the 6 month period and 
onward are considered to be in the maintenance stage (Norcross et ah, 2011). Preserving behavior change and 
avoiding relapse can last a lifetime and is hallmark of maintenance (Prochaska et ah, 2013). 

TTM can transition to a wider social and cultural context, existing as a cycle of stages influenced by an 
individual that takes into account personal characteristics and surrounding communities (Bmg, Conner, Harre, 
Kremers, McKellar, & Whitelaw, 2005). Critiques of the TTM have focused on: social forces eroding active stage 
gains and regaining their power, imprecise relationships between stages and processes, and no account for internal 
validity threats in TTM evaluations due to complexity of human interactions (Bmg et ah, 2005). To address these 
observations of TTM, we posit that implementing a socially constructed process model into a set of instructional 
tasks can alleviate social tendencies of relapse and mitigate credibility issues found in descriptive, interpretive, and 
theoretical validity threats. 

Re-socialization through Alternation 

Re-socialization through alternation (RSA) is a process model occurring in four phases: the construction of 
plausibility structures, segregation from the outside with necessary socializing and therapeutic personnel, 
elaboration of a body of knowledge, and necessary legitimation and nihilations to make sense of greater truths 
(Berger & Luckmann, 1966). These phases are procedural in nature, interconnected, and takes into account 
subjective assessments as well as interpretations of the past and present. Successful asthmatic identification does not 
occur overnight and asthmatic self-identification must be symmetrical from a micro and macro societal perspective 
(Berger and Luckman, 1966). Micro and macro societal perspectives are transposed to objective and subjective 
realities of fixed analysis and self-internalization. For asthmatics, a macro perspective would be fiscal burden due to 
emergency room visits and a micro perspective would be an increase in happiness and reduction in stress due to 
decreased asthma physical limitations. 
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A social base serving as the environment for transformation is the single most important available condition 
during construction of plausibility structures (1966). Emotional dependency of significant others aide in mediating 
the new world to the individual while transformation of subjective reality is still out of focus and in question. 
Plausibility structures must become the individual’s world, with cohabitants of the world that bolster rebellious 
behavior left behind (1966). 

In the scope of asthmatic self-identification, segregation from the outside with necessary socializing and 
therapeutic personnel is more a mental segregation than a physical one. Alternation requires a reorganization of 
conversation to educators, family, and the community that avoids discrepant definitions of reality and ‘backsliding’ 
tendencies (1966). This cannot be done with total success, but protection from disrupting influences until mental and 
physical novitiate stages solidify are suggested. 

Elaboration of a body of knowledge is dialectical and ebbs from educator to individual to society (1966). 
Constructing a body of knowledge is an experiential phenomena in SCR, understanding that perceptions of reality 
are related and negotiated by people, human typifications, significations, and institutions. Large and small group 
discussions on a body of knowledge are common in a constructivist environment, affording individuals the 
opportunity to exercise self-regulation and desire to persevere through tasks. Immersive and dialectical relationships 
to a body of knowledge plays a vital role in increasing ability to test ideas, synthesize ideas of others, and build 
deeper understanding of what they are learning (Reznitskaya, Anderson, & Kuo, 2007; Weber, Maher, Powell, & 
Lee, 2008). 

Necessary legitimations and nihilations throughout the sequence of alternation helps to reinterpret the old 
reality within the context of the new reality (1966). New subjective truths to reinterpret past life experiences 
accommodate for schema change and nihilate negative categorizations of self-identification; for example, a ‘then I 
thought, now I know’ contemplation of past significant events aide in harmonizing the remembered with the 
reinterpreted past (1966). 

In re-socialization the past is reinterpreted to conform with present reality, with concomitant stages 
allowing for transitional fellowships between educator, individual, and community. Criticisms to constructivist 
learning include ignoring objective measures and being destructive to introverts who don't perform well under social 
pressure. Criticism also focuses on learning as a socially negotiated process, which is time exorbitant in achieving 
collaborative and consensual understanding (Merrill, 2009). 

Gap of Asthma Education Program Practice 

According to Eastwood and Sheldon (1996) there is sparse research on organized approaches to deliver 
asthma care. They also maintained that asthma education programs lack organization and devoid of multiple, 
corrective-based instructional strategies being implemented while also not sufficient attention on providing 
healthcare professionals with the appropriate skills and tools needed for educating patients (Eastwood & Sheldon, 
1996). Bender (2002) offered that two barriers to patient adherence to treatments were not only the overly 
complicated treatment plans, but also patients did not perceive the treatments as being beneficial. Barriers within the 
education programs included scheduling difficulties, little training on patient education for clinicians, rotating care- 
givers (Bender, 2002). Patient barriers to adherence included not understanding treatment, lacking sufficient faith in 
care-givers, and perhaps, cognitive and motivational deficiencies (Bender, 2002). Because of such barriers, there 
appears to be a transferal gap in patients’ knowledge, skills, and attitudes from an educational setting to the 
lebenswelt, or experienced lifeworld. Changes in health behaviors by the patient need to occur throughout an asthma 
education program and transfer from these education programs into their daily experiences for a sustained time 
period in order for adherence to occur. Current educational interventions related to treatment, clinician, and patient- 
related barriers to adherence appear to lack rigor in aligning instructional context to performance context. However, 
we posit that integrating the TTM and RSA for asthma education offers a modern framework for instructional 
strategies to promote self-efficacy and allow for processes of change and adherence to occur within patient health 
and wellness behaviors. 


Alignment for Asthmatic Self-Identification 

Precontemplation through Construction of Plausibility Structures 

The identification of plausibility structures are established through family, friend, and community 
awareness in conjunction with asthma educational programs. The asthma educator serves as the ego ideal during this 
stage, acting as initiator of all educational activities to an asthma patient who is skeptical in acknowledging a 
problem with self-management. During this stage there is no sincere effort in deviation from the status quo and 
attentional focus on correct asthma medication administration is fraught with habitual excuses for non-adherences 
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(Prochaska, Norcross, & DiClemente, 2013). The asthmatic patient is completely naive to their own self-identity, 
with a complete incoherence to associated health risks and problems. For the asthma instructor this is a time to build 
a basic understanding of the asthma process and treatment approaches. Bender suggested strategies for promoting 
adherence emphasizing basic rationale for following the medication regimen and chunking educational sessions into 
small segments to prevent learner fatigue (2002). Broers et al. used a similar approach, implementing basic agenda 
setting through honest and open discussions, a basic willingness of the patient to discuss the topic, and maintaining 
an open approach to any questions the patient would discuss (2005). 

Contemplation and Preparation through Socializing and Therapeutic Personnel 

Asthma identification encompasses self, familial, friend, and educator involvement, and family and friends 
are primary external socializing agents during asthma identification. Collaborative education during this stage assists 
with awareness raising of adjacent habits aiding or detrimental to asthma control. Ambivalence towards the disease 
and consequential weighing takes place during the contemplation stage, and exploring a cost-benefit analysis from a 
monetary and quality of life perspective is suggested (Borrelli, Reikert, Weinstein, & Rathier, 2007). 

The contemplation stage is also the first time to explore motivation and readiness for changing. This is a 
time of budding motivational involvement by the asthma patient in which identification is consequentially tied to 
costs from a monetary and vitality perspective; furthermore, it is not uncommon for conceptual conflict during this 
stage to enhance accommodation efforts tied to costs (Nussbaum & Novick, 1982). It would stand to reason that 
motivational inquiries into these two perspectives would be beneficial in advancing the identification process, but 
scant information is available on the topic. Motivational interviewing is a known strategy to assess motivation and 
readiness to change in terms of medication adherence (Riekert, Borrelli, Bilderback, & Rand, 2011). 

During the preparation stage there is a commitment to change and readiness to submit to an asthma action 
plan regimen in earnest. The preparation stage is characterized by goal setting and skills training with a focus on 
active knowledge building through exercise examples, continuous feedback mechanisms, use of analogies, and 
chunking of like schema objects. Goal setting is variable and based on learner analyses, but commonalities of a 
uniform checklist(s) attests to skills correctly performed in terms of meter dose inhaler usage and determination of 
possible triggers (Bruzzese et al., 2004). Attitudes and self-efficacy rather than knowledge have a significant impact 
on knowledge and the preparation stage is a time for the educator to stoke the flames of concomitant interest in 
asthma identification through increasing one’s belief in their capabilities to successfully execute a course of action 
(Scherer, 2001; Ngamvitroj & Kang, 2007). This stage is preparatory to the action stage as long as self-efficacy level 
is sufficient. 

Action through Elaboration of a Body of Knowledge 

The elaboration of knowledge through group and individual reflective and reiterative exercises is a time to 
empower confidence that self-management strategies can control asthma symptoms. Rewards to sustain motivation 
during this stage are common, as emotional stress may influence asthma adherence through neuronal pathways 
(Campbell et al., 2006). Support from family is essential during the action stage due to constant tension between the 
primary and blossoming re-socialized identification. Due to this tension, the action stage is a common locality for 
partial or total relapse back into the primary socialization identification. Self-care skills and reinforcement through 
scaffolding aide in bridging gaps in knowledge while helping reduce partial and goal relapses (Vina, 2005). 

Maintenance through Necessary Legitimations and Nihilations 

Making sense of the journey leading to asthma self-identification during a socially constructed process and 
achieving self-autonomy is contingent on maintaining the legitimations developed throughout the first four stages 
such as improvement in quality of life, decrease in school or job absences, reduction in primary care visits, and 
decrease in clinic and hospitalization associated fiscal demands. There is a sizable literature gap in studies 
addressing asthmatic maintenance programs that could be of benefit thousands of asthmatics who experience 
knowledge erosion and subsequent frequent readmissions to clinics, emergency departments, and hospitals. This 
integrative asthma identification change model is intended to promote and expand long-term memory and 
maintenance motivation, but at this time applicability of the model is needed for substantiation. Table 1 provides an 
incorporated view of social constructive strategies that are consummate with the integrated TTM and RSA 
framework. 
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Table 1 . Integrated TTM and RSA framework 

Recommendations 

Asthmatic self-identification follows a holistic approach of known successful socially constructed 
instructional and motivational strategies to posit a modern change model for behavior. Openness in communication, 
reflective and reiterative exercises, built-in feedback mechanisms to exercises, and a relaxed atmosphere are good 
starting points for any asthma education program striving for asthma patient self-identification. It is recommended 
that further research on constructivist models of asthma education be carried out, with emphases on: evaluation 
scales assessing influence of monetary, vitality, and associative risks of asthma; self-care skills and reinforcement 
strategies designed to bridge gaps in asthma knowledge and reduce relapses; and addressing asthmatic maintenance 
programs designed to limit cognitive erosion and reduce readmissions to clinics, emergency departments, and 
hospitals. 
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Abstract 

This research studied the design of a structured peer tutoring program to scaffold the development of student online 
learning readiness. Structured peer tutoring interventions were designed and implemented in an asynchronous 
orientation course. Pre-designed intervention messages, trainings, and consultations were provided to the student 
tutors. The implementation process and perception of student tutors were investigated. The findings lay a new 
ground for providing more meaningful and effective support to student success in online learning environments. 

Introduction 

As incubators for community of practice, online learning environments indicate a potential to accelerate 
learning with the abundant web-based resources and through peer learner connections (Boud, Cohen, & Sampson, 
2001; Casey et al., 2014). Peer tutoring has been identified as beneficial to learning (Ali, Anwer, & Abbas, 2015). 

To sustain the quality of tutoring, appropriate selection of peer tutors has been suggested (Van Rosmalen et al., 
2008). More importantly, trained and structured tutoring is recommended for both the learners and the student tutors 
(Evans & Moore, 2013; Fuchs, Fuchs, Bentz, Phillips, & Flamlett, 1994; Weyrich et al., 2008). This research studied 
a design of structured peer tutoring in an asynchronous orientation course for developing student online learning 
readiness. A student peer tutor was trained to understand the purpose and procedure for deploying the pre-designed 
interventions. This paper presents the problem, purpose of the study, design solution, and preliminary findings with 
discussion. 


Context and Problem 

As students enrolled in fully web-based courses face the evolving technologies in online environments and 
the paradigm shift of self-regulated learning in an anytime-networked world, they need preparation to be successful 
in online learning. The preparation includes communication and interaction literacy, technology literacy and skills, 
time management strategies, formal and informal learning strategies regarding online and digital facilitation, and 
capability for seeking help when encountering technical difficulties (Liu & Adams, 2015). To prepare students with 
online learning readiness, a self-guided asynchronous orientation course with scaffolding was designed in 2013- 
2014 academic year in a Master’s-level University along the East Coast of the United States. Undergraduate and 
graduate students enrolled in a fully online for-credit course in the summer sessions were invited to participate in the 
orientation. The course was built in the same learning management system (LMS) as the for-credit courses for ease 
of access and just-in-time support (Er, Ozden, & Arifoglu, 2009; Ullmann, 2009). 

Even though this generation has grown up with bits and dots, students don’t sufficiently understand the 
need of preparing for the online learning environments (Bozarth, Chapman, & LaMonica, 2004; Salmon, 2011). 

This was reflected in the formative evaluation in the first iteration of implementing the orientation course. A major 
activity in the orientation was a self-awareness assignment that required students enrolled in a fully online course to 
plan the 168 hours in a week, to estimate the hours reserved for the online course. A worksheet with an example 
served as the assignment prompt. Many students who submitted the completed worksheet did not estimate sufficient 
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time for online course participation. Due to the limitation of resources, the assignments used automatic grading in 
the LMS with minimal human intervention. Anecdotally, an instructor teaching a summer online course requested to 
be a teacher in the orientation site and pointed out how few hours one of his students initially planned for an online 
course. A follow-up discussion between the instructor and the student provided an informative path to the student’s 
successful completion of the online course. 

The anecdotal finding and previous literature (Mostow, et. Al. 2003; Rodicio, & Sanchez, 2012) led to the 
revised version of the orientation course, which solves the problem of how human interventions can be designed and 
efficiently included in the orientation course. The problem would need to be answered with the consideration of 
three factors, including efficiency of human resources, technology affordance, and impact on student performance 
(Reeves, Herrington, & Oliver, 2005; Richey & Klein, 2007). 

Purpose of Study 

Based on a review of literature and the consideration of efficiency and affordance, the redesign with 
structured peer tutoring interventions was implemented in the 2014-2015 orientation course. The procedure for 
designing the peer tutoring interventions was documented. A student peer tutor received training and deployed the 
interventions. Per requirement and approval of Institutional Review Board, the student also granted the consent of 
being a research subject in this study. The current paper focuses on probing answers to the following questions: 

• How can student peer tutoring interventions be designed and implemented into an orientation course? 

• How does a student tutor perceive the role in providing the pre-designed interventions in an orientation 
course in order to prepare her/his peers' readiness for online learning? 

Design Solution 

There were four major learning units in the orientation course that invited students’ voluntary participation 
and interaction within the LMS. These were designed to acquaint students with the essential skills and strategies to 
be ready for fully online courses. In the redesigned 2014-2015 version, human interventions were designed with 
unique features and included in two of the units, as demonstrated with Figure 1: 
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Figure 1. Peer tutoring interventions 
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Previously, in the Time Management Strategies unit, students reviewed an example worksheet for planning 
the hours in a week and then downloaded a copy to estimate the number of hours reserved for taking a fully online 
course. The assessment was automatically performed using the LMS grading feature. In the redesigned version, the 
student tutor was given a comment message template with the recommended number of hours that should be 
reserved each week for a fully online course. When the tutor detected a low estimate, she posted the comment in the 
grading feedback area and subtracted a few points from the 100-point assignment. 

In the Synchronous Online with Blackboard Collaborate unit, students practiced launching a synchronous 
session within the LMS. This task was known to be challenging at the beginning of an online class. The student tutor, 
guided by the primary investigator of this research, developed a customized screen cast tutorial to demonstrate the 
process from a student perspective. The accompanying activity allowed students to post comments after completing 
the task, which further disclosed additional questions from the students. The student tutor was trained to provide 
answers or directions, depending on the comments submitted. 

The design of intervention messages drew from components in Salmon's e-Moderating Model (De Smet, 
Van Keer, & Valcke, 2008; Lisewski & Joyce, 2003) and Motivational Message Support System (Visser, 1998). For 
instance, "[distance students] should be helped to define priorities and to manage time" (Visser, 1998, p.59). The 
composition of these messages took into consideration elements such as access to relevant information, gentle 
reminder to prioritize online class over other online presence, timely and encouraging feedback, familiarization with 
tools, and facilitation of the process for knowledge construction. An example message can be found from this link . 

The design also included the pre-intervention training for the student peer tutor and on-going check points 
in the intervention process. The student tutor was provided information and training about the interaction 
mechanism in the LMS, methods of deploying intervention messages, and domains of content such as customized 
access to synchronous online web conferencing and technical support for computer configuration. The purpose of 
the intervention messages were explained and discussed during the training. Scenarios were discussed to prepare the 
student tutor for engaging with peer students in the orientation course. Consultation from the instructional designer 
was available during the tutoring if decisions or modifications were needed. 

Data Collection and Analysis 

Data were collected from triangulated sources, with the informed consent from the student peer tutor. The 
first source consisted of the extensive dialogues with the student tutor during the training and consultation sessions 
that were provided to the student tutors. Thorough notes were taken during these sessions. Transcripts were 
collected when emails or online discussion were used. The second source came from the frequency and types of 
messages that the peer tutor deployed. These included the peer tutor's feedback on student assignments submitted in 
the orientation course, responses to discussion postings, and communication messages via the LMS. The third 
source of data was collected with a semi-structured post-tutoring interview. "Language is viewed as the primary 
symbol system through which meaning is both constructed and conveyed" (Rossman & Rallis, 2003, p. 97). The 
language captured during the interview provided a rich reflection about the peer tutoring experience. The interview 
was audio-recorded and transcribed. 

Applied thematic analysis was utilized to categorize and analyze these data. This type of analyses moves 
“beyond counting explicit words or phrases and focus on identifying and describing both implicit and explicit ideas 
within the data, that is, themes" (Guest, MacQueen, & Namey, 2011, p.10). Transcripts were analyzed carefully and 
repeatedly line by line. Coding techniques with colors and/or numbers were used. 

Findings and Discussion 

To prepare student readiness for online learning, structured peer tutoring can be designed and implemented 
in an asynchronous orientation course. Different from existing literature on peer tutoring, learning in this course is 
not attached to a specific subject area. The acquisition of strategies and skills in this course, however, will be the 
means to success in learning domain-specific knowledge through the mediation of technologies. Strategies learned 
in this course, such as time management strategies that are essential for planning a day when taking a fully online 
course, will benefit students holistically. 

In this study, a student tutor received training about the methods of deploying structured peer tutoring 
interventions, the contextual information and guides about supported technologies for online learning, and 
appropriate directions for technical issues. Given the technology affordance (Reeves, Herrington, & Oliver, 2005), 
design considerations were applied to train the student tutor to use the pre-designed intervention messages and make 
decisions about timing for deployment. As the major components of a specific type of design-based research 


127 



(Reeves et al., 2005; Richey & Klein, 2007), the training, pre-designed messages, tutoring process, and perception of 
the student tutor were studied to answer the research questions. 

To lead to the gradually refined design of this program (Reeves et al., 2005), understanding the perception 
of student tutors in this particular type of peer online tutoring was instrumental. This research explored the 
experience of the student tutors (Rossman & Rallis, 2003), and reached several exploratory conclusions, including 
the student tutor’s perceived confidence in performing tasks in the LMS, the value of human interaction, and the 
need of more support to mandate students’ participation in the orientation courses. The data analysis results of this 
one case also provided positive feedback about the design of structured peer tutoring. 

As the findings indicated, the design with the pre -defined structure, training and accompanying 
consultation were essential to student peer tutoring in this kind of online orientation courses. The student tutor 
expressed confidence in deploying the messages based on the training and knowing that the supportive consultation 
was always available. The awareness of various time management perceptions of peers was an eye-opener for the 
student tutor. The process also helped the student tutor develop empathy for peers who continued to take online 
courses while having other commitments such as family commitments and jobs. Finally the student tutor confirmed 
the importance of human interaction through the intervention process. Although auto-generated messages could 
have been used with an online mechanism, students would take it seriously if the comments were from a real person. 
This further confirmed what had been found in the previous literature about the importance of a human tutor in a 
technology-mediated learning environment (Mostow, et. Al. 2003; Rodicio, & Sanchez, 2012). 

Based on the results of this design case and understanding of the student peer tutor’s perception, the impact 
and lessons learned can be used to "determine transfer effects of the program" (Richey & Klein, 2007, p. 59). The 
researchers are looking forward to the future extension of this study. In the coming iterations, follow-up interviews 
with students who receive the intervention messages can be conducted to find out more about the impact of the peer 
tutoring on their learning performance in the for-credit online courses. Frequency of students’ returning to the 
orientation course site for support information can be collected and analyzed before the course concludes, which can 
provide background information for making decisions about support allocation. Continuing this type of research can 
lead to more effective paths for designing student tutoring programs in facilitated online environments. 
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Abstract 

This study investigates the potential of a tablet-based note-taking application (TbNA) as digital science 
notebooks to enhance pre-service teachers’ science practices. Twenty-seven pre-service teachers in an elementary 
science methodology class participated in this study. During one semester, pre-service teachers used a tablet-based 
note-taking application as their digital science notebook. Their notebook entries and responses to surveys regarding 
their experiences with the TbNA were collected. The study discusses how pre-service teachers made use of tools 
within the TbNA. 

Keywords: Elementary, Science Practices, Technology Implementation 

Introduction 

Scientific inquiry, which refers to “the diverse ways in which scientists study the natural world and propose 
explanations based on the evidence derived from their work” (National Science Education Standards, 1996, p.23), 
has been a major emphasis of modern science education. Researchers have suggested that one approach to doing and 
learning inquiry-based science, is for educators to consider language and writing as essential parts of science, as 
real-world scientists engage in reading, writing, and talking as part of their everyday scientific practice. For 
example. Yore, Florence, Pearson, and Weaver (2006) explain that the writing process is an integral part of science 
and is an extension of the inquiry and interpretation processes. Baker et al. (2008) highlight that writing skills are 
important when using inquiry-based approaches in the science classroom. Howes, Lim and Campos (2009) also 
claim literacy practices should be central to science teaching since reading, writing, and speaking are foundational to 
the work of professional scientists, as well as to the comprehension of ideas by nonscientists. 

Many teachers at the elementary level incorporate writing in science through the use of science notebooks 
(Baxter, Bass, & Glasser, 2001; Fulton & Campbell, 2014; Fulwiler, 2007; Gilbert & Kotleman, 2005; Shepardson 
& Britsch, 2004; Worth, Winokur, Crissman, Heller-Winokur, & Davis, 2009). It can be, however, a challenging 
and complex task for primary-aged students to engage in writing focused on the scientific practices, such as 
constructing explanations and obtaining, evaluating, and communicating information. Thus, the role of teachers is to 
provide explicit instruction and support mechanisms to help students develop their scientific writing ability. Part of 
learning how to do this is to have had experience using science notebooks themselves, such that they can understand 
the wide variety of strategies they can use to implement them in their own science classrooms (Morrison, 2008). 
Accordingly, science notebooks have been introduced to pre-service teachers through their methods courses 
(Morrison, 2008; Moseley, Ramsey, & Ruff, 2004; Plummer, Zahm, & Rice, 2010). 

Traditionally, science notebooks have been introduced in the format of paper-based notebooks. Common 
notebooks range from composition books, spiral notebooks, three ring binders with loose-leaf paper and section 
dividers, to three -prong paper folders (Cox, 2012; Fulton & Campbell, 2014). Meanwhile, studies have suggested 
the potential benefits of integrating Information Communication Technology (ICT) for science learning. Such 
benefits include the encouragement of communication, collaboration in science research activities, collection of 
scientific information, and interaction with multimedia resources (see Bingimlas, 2009). Given the theoretical and 
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empirical support for integrating notebooks into scientific practice and the popularity of mobile technologies, 
researchers should explore the potential of technology to support and possibly enhance the use of science notebooks. 
With this in mind, our research investigated how a specific digital technology — a tablet-based note taking 
application — influenced pre-service teachers’ scientific practices in a science methods course. The study examined 
how pre-service teachers use the tablet-based notebook application and, ultimately, their perception of digital 
science notebooks. 


Conceptual Framework 
Science notebooks for inquiry-based Science 

Due to the fact that the field of science education interprets the term “inquiry” in many different ways 
(Barrow, 2006; NRC, 2012), A Framework for K-12 Science Education (NRC, 2012) articulated the scientific and 
engineering practices as a way “to better specify what is meant by inquiry in science and the range of cognitive, 
social, and physical practices that it requires” (p. 30). These practices include: 

(1) Asking questions and defining problems 

(2) Developing and using models 

(3) Planning and carrying out investigations 

(4) Analyzing and interpreting data 

(5) Using mathematics and computational thinking 

(6) Constructing explanations and designing solutions 

(7) Engaging in argument from evidence 

(8) Obtaining, evaluating, and communicating information (p. 42) 

As these components show, asking questions, collecting and documenting data for evidence, formulating 
and evaluating explanations, and communicating those explanations are important aspects of science, many of which 
involve writing. Consistent with this view, Yore et al. (2006) argue that scientists write to develop claims and 
arguments about that which they are studying, stating, “scientists use writing to create permanent records to 
establish their priority for discoveries and as documented sources for reflection, analysis, and evaluation” (p. 113). 

Consequently, the importance of writing has been underscored in inquiry-based science education. Baker et 
al. (2008) argue that writing promotes critical-thinking skills and the construction of vital scientific concepts in order 
to challenge ingrained misconceptions. Explaining that science notebooks have been used to document scientific 
discovery, Fulton and Campbell (2014) argue they are effective tools in the classroom. That is, writing in notebooks 
makes science experiences more meaningful and authentic for students as they observe, record, and reflect upon 
what they have learned. Reid-Griffin, Nesbit and Rogers (2005) also argue that science notebooks allow students to 
act as scientists by sharing what they have learned through the notebook and through group discussions. Further, 
researchers have shown when students use their notebooks in a manner similar to that of scientists there is a strong, 
positive correlation between their performance of scientific practices and their understanding of the science content 
(Aschbacher & Alonzo, 2006; Ruiz-Primo, Li, Tsai, & Schneider, 2010). 

Teachers’ Use of Notebooks 

While the goals of science notebooks as a means to develop students’ scientific practices have been 
discussed, the efforts to properly introduce science notebooks to pre-service and in-service teachers is an area of 
concern. Recognizing the difficulties and challenges that students, especially primary-aged students, face when 
writing in science, research has emphasized the importance of teachers’ supports and guidance related to science 
notebooks. Aschbacher and Alonzo (2006) show that teachers’ use of notebooks influences what students write in 
them and how informative these entries are with respect to student understanding. Similarly, Butler and Nesbit 
(2008) found that teacher’s directions related to using science notebooks influenced the quantity, quality, and 
organization of students’ entries. Based on this, Butler and Nesbit describe strategies for improving students’ writing 
and deepening their conceptual understanding through the use of Science Notebooks. 

Accordingly, the needs for in-service and pre-service teachers’ experiences with science notebooks have 
been emphasized. Lewis, Dema, and Harshbarger (2014) found that pre-service teachers resisted the use of science 
notebooks, as they saw it as an organizational tool initially. However, the more they used the notebook the more 
they started to view it as a teaching resource that mirrored elementary students’ use of science notebooks as 
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documentation and a product of their science learning. Morrison (2008) also underscores it is important to introduce 
both pre-service and in-service teachers to proper uses of science notebooks in order to encourage the use of science 
notebooks in elementary classrooms. Morrison (2005), specifically introduces science notebooks as a tool for 
formative assessment, and argues “it is crucial to provide preservice teachers with experiences using the notebooks 
themselves as students and to require them to visualize and evaluate the positive aspects of the use of notebooks in 
their future classrooms (p.19)”. 

Digital Science Notebooks 

Technology integration into classrooms has become an important topic in education. How technology can 
facilitate, support, and improve teaching and learning are continuously discussed. Given the emphasis on writing in 
science education, the potential of integrating ICTs into science classrooms has also generated attention. For 
example, Osborne and Hennessy (2003) explain that ICTs offer a range of different tools for use in science-related 
activities such as tools for data capture, processing, and interpretation; multimedia software for simulation of 
processes and carrying out virtual experiments; digital recording equipment; and publishing and presentation tools. 
Further, they emphasize that a teacher’s role is critical to creating an appropriate ICT-supported learning 
environment by leveraging effective pedagogical practices. Explaining that ICT can support both the investigative 
and knowledge-based aspects of primary science, Murphy (2003) emphasized the importance of verbal as well as 
written communication — vital aspects of children constructing meaning — noting that ICT can provide opportunities 
for children to engage in effective communication. Rappolt-Schlichtmann et al. (2013) have investigated the 
potential for a web-based science notebook design using the Universal Design for Learning framework. By 
conducting research in fourth grade science classes, they found a positive impact of the web-based science notebook 
on student learning. The impact included improved science content learning outcomes, student performance, high 
levels of interests, and feelings of competence and autonomy. 

As a means of integrating ICT into science writing, the researchers introduced a tablet-based note-taking 
application (TbNA) as a digital science notebook in a science methods course. The purpose was to assess the interest 
of pre-service teachers in using it as a learning tool in an undergraduate-level course. While it is our goal to 
investigate how use of a TbNA can support and enhance pre-service teachers’ conceptual and pedagogical 
understandings about using a science notebook to promote writing related to the scientific practices, this study 
focused on examining the usability of TbNAs as notebooks in a science methods course. 

Method 

This research was conducted at a public university in the Pacific region. Twenty-seven pre-service teachers 
(n = 27) enrolled in an elementary science course participated. A tablet computer was provided to all pre-servie 
teachers at the beginning of the semester. The instructor of the course introduced the TbNA as a “digital science 
notebook”, and the pre-service teachers used the TbNA as part of their science practice for the entire semester. 

At the end of the semester, the pre-service teachers responded to a survey about their use of the TbNA. In 
addition, they submitted a written reflection on the use of science notebooks and their notebook entries were 
collected. For the survey responses, descriptive data analysis was conducted to examine the use of specific technical 
aspects of the TbNA. The written reflections were analyzed using a constant comparative method (Glaser & Strauss, 
1967) to uncover emergent themes. The purpose of the analysis was to determine the pre-service teachers’ perceived 
benefits and challenges of using digital science notebooks. The pre-service teachers’ notebook entries were analyzed 
using content analysis following the methods of Ruiz-Primo and Li (2004) to identify the focus of each entiy. This 
paper presents the findings from the pre-service teachers’ survey responses. 

The software used by the pre-service teachers as a science notebook was Notability. This application was 
selected based on the criteria of pre-identified technical features. These features include word processing, 
handwriting recognition, audio recording, the taking and inserting of photos, and sharing notes. 

Results 

A total of seven questions, including one multiple-choice question and six open-ended questions, were 
given to the pre-service teachers. The first question was a multiple choice question asking the pre-service teachers 
which tools available in Notability they used while recording in their science notebook. All of the possible tools in 
the application were listed as options, and the pre-service teachers were asked to check all the tools they used. Table 
1 presents the frequency of the pre-service teachers’ responses. 
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Table 1 

Pre-service Teachers’ Use of Note Taking Application Tools 


Tools 

Number of Responses 

% of Responses 

Erasing 

25 

93% 

Taking a photo 

25 

93% 

Typing 

24 

89% 

Drawing/Coloring 

23 

85% 

Handwriting 

22 

81% 

Highlighting 

22 

81% 

Sharing (Google Drive) 

21 

78% 

Cutting/Pasting 

18 

67% 

Zooming (Magnifying glass) 

16 

59% 

Changing paper 

15 

56% 

inserting figures 

13 

48% 

Palm resting 

9 

33% 

Audio recording 

8 

30% 

Inserting Web clip 

8 

30% 

Adding stickies 

8 

30% 


The open-ended questions asked the pre-service teachers about their most and least favorite feature as well 
as the feature they found most difficult to use. The following section summarizes the pre-service teachers’ 
responses. 

Regarding the most liked features, the pre-service teachers listed various tools including the magnifying 
glass, palm rest, drawing, highlighting, handwriting, and taking a photo. Among these, the tool that most of them 
chose was taking photos. Nine out of twenty-seven pre-service teachers (33%) chose taking photos as their favorite 
tool. Below are some of the pre-service teachers’ comments on why they liked using the photo tool: 

• This was good because I was able to have pictures with captions of explanations of what I was doing 
for that specific task. 

• I liked this tool the most because I was able to put my hands-on experiences directly in my book where 
I was already writing about what I was doing in class. This gave me a visual representation of what I 
was trying to write about in my book and also gave me a point of reference in future conversations. 

• I liked being able to take photos and put it in our notes the best because it gives us record of a visual of 
what we were doing and what our notes refer to. 

As shown by the pre-service teachers’ comments, it was suggested that they liked taking photos as a tool 
for visually recording what they saw and did related to their various lessons. 

The second most liked tool was handwriting, which was selected by eight pre-service teachers (30%). 
According to the comments, students seemed to feel comfortable using the handwriting tool. They commented that 
they liked the handwriting tool because it was easy and flexible. Below are the pre-service teachers’ comments: 

• Because I wasn't limited to a restricted keyboard 

• I was able to take as many notes as I wanted to. I didn't have to worry about wasting paper, and it was 
still my own handwriting. I could erase as much as I wanted and 1 loved it! 

• I like being able to write on my notes as the action of writing helps me to remember. 

• Writing was very fun and helped fill in forms, worksheets and powerpoints to take notes. 

Meanwhile there were two pre-service teachers who chose the handwriting tool as their least favorite tool. 
They suggested that it was difficult to write on the small screen, and that zooming in and out was not so convenient. 
Also, inserting web clips was chosen as the least favorite tool. Pre-service teachers reported that they couldn’t figure 
out how to use this feature. Furthermore, palm resting and the magnifying glass, and audio recording were also listed 
as least favorite tools. Despite these struggles and least favorite tools, overall the pre-service teachers seemed to be 
satisfied with the TbNAs as their notebooks. Thirteen pre-service teachers (48%) responded that there was no tool 
that they did not like. 
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Similarly, the pre-service teachers chose taking photos (8 responses), typing (6 responses), and handwriting 
(4 responses) as the most helpful tools to record data. Similar responses continued for the question about the most 
frequently used tool. What was interesting was that typing (8 responses) and handwriting (4 responses) were also 
chosen as difficult tools for data recording. The pre-service teachers explained that typing was hard for the reasons 
below: 

• I was limited to the characters and felt pressured and felt as though my data was not being recorded 

• Again, typing was hard on the iPad because I'm used to typing on an actual keyboard instead of using 
the keyboard on the iPad. 

• I don't have a keyboard attached to my iPad, so it took longer for me to type than to write. 

Importantly, pre-service teachers were asked if they saw any advantages to using a digital notebook 
compared to a paper-based notebook. Their responses could be categorized in three ways: 1) availability of various 
multimedia, 2) clearer and easier organization, and 3) easy accessibility and sharing. In support of these benefits, the 
pre-service teachers shared the following comments: 

• Able to privide [.sic] pictures and data and audio recording 

• Being able to insert pictures right away and having an easier time submitting things electronically 

• A digital notebook enables me to take pictures, draw pictures, and it is easier to refer back to instead of 
a stack of papers 

• Digital notebooks are much more versatile and can easily incorporate many notes and pieces of 
information. 

• It's more variety and accessibility in digital notebooks 

• I lose paper all the time. If I let someone else use my notebook I don't have access to it. I can add 
pictures. 

• With a digital notebook, you can save you notes in multiple places, which makes it harder to lose them. 
It also makes sharing your ideas with your peers and teachers easier because you could email them 
what you have been doing instead of making physical copies. 

• It is neater and more organized. 

While many pre-service teachers listed the technical features as the advantages of the digital notebook, a 
few comments seem to suggest that they were able to make connections to the possibility of enhanced science 
learning: 

• Students can place class photos and videos directly in the notebook to give reference and deepen their 
understanding rather than rely only on what they write on paper. 

• The advantage of using a digital notebook versus a paper-based notebook is the fact that we could 
insert photos and videos to our notes. It's nice to have a visual for our notes to be able to reflect or 
make connections 

• The students are more engaged in technology. The studetns [.sic] can record their data easier. 

On the other hand, several pre-service teachers listed the disadvantages of using digital notebooks. These 
disadvantages focused on technical challenges such as loosing data, technology failures, and possible distractions, as 
shown below: 

• Technology failing and losing data in the notebook. 

• it getting lost or deleted on accident, its not tangible, remembering and searching for information that 
you wrote in different days 

• You get distracted because an iPad or tablet offers more than just the app. 

• Computer crashes, and you have nothing to back it up with. 

• technology will and can fail 

Lastly, pre-service teachers were asked which format they preferred as a science notebook. Eighteen pre- 
service teachers (67%) chose the TbNA and the other nine (33%) chose paper-based notebooks. Apparently, pre- 
service teachers choosing the TbNA as their preferred notebook gave similar reasons to the advantages described 
above. On the other hand, those who chose paper-based notebooks explained that they liked paper-based notebooks 
because they were more comfortable with them. Interestingly, a couple of the pre-service teachers stated that they 
remembered better with the paper-based notebook: 
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• I'm very comfortable with paper and pen writing, I best remember things when I handwrite them 

• I don't remember things if I type them out. 

• I like to write things down because it helps me to remember things better. 

• I prefer taking nots [.sic] on paper notebook because I'm used to being able to write with pencil and 
paper and it is just more convenient for me. 

Discussion/Implication 

The aim of this study was to examine the potential of a TbNA to act as a digital science notebook. As the 
first step, a TbNA was introduced to 27 pre-service teachers in a science methods course. After using the TbNA as 
their science notebook, the pre-service teachers responded to a survey regarding their use of the TbNA. Based on 
their responses, most of the pre-service teachers used the majority of the tools available in the TbNA without 
difficulty. That is, tools for taking a photo, handwriting, typing, drawing, highlighting and sharing were used by 
most of the pre-service teachers. However, it appears that only a few tried to record audio, add “stickies”, or insert 
web clips. In terms of their favorite tools, it was not surprising to find that taking a photo was the most popular 
feature. The pre-service teachers seemed to value the capability of recording the process and activities of their 
learning in a visual manner. This also appeared in their responses about the advantages of digital notebooks. Several 
of the pre-service teachers explained that being able to take photos and record audio and video might be advantages 
of using digital notebooks compared to paper-based notebooks. 

Interestingly, there seems to be clear differences in the pre-service teachers’ preferences of how to write 
using a TbNA. While there were eight who chose the handwriting feature as their favorite tool, some pre-service 
teachers clearly stated it was difficult to use the handwriting tool on the TbNA. Apparently, some preferred paper- 
based notebooks, as they explained that they liked handwriting with a pen and paper. It would be interesting to 
examine how their experiences of using the handwriting tool in the TbNA compared to using a real pen and paper. 
Even though it is evident that with the advancement of technology, handwriting on touchscreens has improved, there 
seems to be room for further improvement before more people feel comfortable with this technical feature. 

Overall, the pre-service teachers seemed to have a positive experience of using the TbNA as their science 
notebook. Even though there were some concerns related to technical issues and challenges of using the TbNA, in 
general the pre-service teachers were able to see the benefits of the TbNA; availability of various multimedia, clear 
and easier organization, and easy accessibility and sharing. 

Although these results are encouraging, the researchers recognize the limitations of the study. First of all, 
what is presented in this paper is descriptive data based on the pre-service teachers’ survey responses. What they 
reported might not directly reflect what they actually did with and how they actually feel about the TbNA. The tools 
the pre-service teachers actually used for their writing and how they used them should be looked into to further 
understand the uses of various tools in the TbNA. Second, focusing on the pre-service teachers’ use of the tools in 
the TbNA, minimizes how such features can support and enhance the development of scientific practices, especially, 
writing. To do so, it is necessary to analyze the pre-service teachers’ science notebook entries. Since they shared 
their entire notebooks with the instructor of the course, using Google Drive. Analysis of their entries is currently in 
progress. Lastly, the study will continue by following the now teachers into their classrooms to see how they 
implement science notebooks, and more specifically digital science notebooks, into their science instruction. 
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Abstract 

Perspective taking is the ability to put oneself in the place of others and recognize that other individuals 
may have points of view different from one’s own (Johnson, 1975). In the field of psychology, education, business, 
and law, perspective taking has been studied as an intervention that presumably affects out-group stereotypicality 
and valence. Conflicting reports, however, have been made regarding how taking the perspective of others may 
influence individuals’ already held stereotypes and attitude toward others. Thus, in this study I designed an 
experiment, which investigates the rather contradicting reports about the effect of perspective taking on 
stereotypicality and valence. Though sample size and test methods were somewhat primitive to make concrete 
conclusions, this study shows that the effect of perspective taking on schemata (e.g. stereotypicality and/or valence) 
may vary by individual differences in social and cultural experiences as well as perception. 

Introduction 

The Framework for 21 st Century Learning acknowledges collaboration as one of the essential skills for 
success in today’s world (Partnership for 21 st Century Skills, 2009) because it enables people and places to be 
interconnected and interdependent. It recognizes that “the world is composed of people who look different, think 
differently, have different belief and value systems, and that we can no longer be hermits and live in complete 
isolation” (Gardner, 2008, p. 12). Accordingly, in education, 21 st -century curriculum and instruction must be aligned 
to produce a support system that produces collaboration among different people (Partnership for 21 st Century Skills, 
2009). Nonetheless, it is known that individuals’ persistence in maintaining certain stereotypes as well as bias on 
others can harm collaboration among people who may differ. In order to facilitate collaboration, it is recommended 
that 21 st -century curriculum and instruction should be designed to address such stereotypes and biases. 

According to Banaji and Greenwald (2013), “the behavioral patterns human acquire, including those 
involved in identity formation, have less rigidity or permanence compared to other species” (p. 127). This implies 
that human brains are relatively flexible and, therefore, are able to unlearn stereotypes and biases towards other 
groups that hold different identities and ideologies. Galinsky and Moskowitz (2000) argued that the constructive 
process of considering and understanding another person’s perspective - perspective-taking - plays a role in 
“debiasing” such prejudice. That is, individuals’ ability to not stereotype others and think positively about them may 
be enhanced through perspective taking. The mechanism behind this is that by encouraging thoughts towards the 
out-group to be more “selflike,” perspective taking reduces the negative and stereotypical attitude towards out-group 
(Galinsky and Moskowitz, 2000; Davis et ah, 1996). Production of assorted civic curriculum, such as “Fair Play,” 
that utilize perspective taking in counter-biasing and stereotyping have followed. 

Nonetheless, it is reported that perspective taking, in actuality, may produce some unintended negative 
consequences. As said by Skorinko and Sinclair (2013), when the person being observed appears to be highly 
consistent with the group stereotype (e.g., an elderly man who is frail and stooped), perspective taking can lead to 
increased stereotyping. More specifically, when individuals observe a person who matches their previously held 
stereotypes about the person they “use the salient stereotypic information as a theory with which to generate the 
thoughts and feelings of the target (p. 17).” Rather than inhibiting the effect of stereotypes, such process would allow 
more stereotypic content and negativity to seep in. Accordingly, careful examination of the effect of perspective 
taking is required before the actual application of such activities in the field. Thus, in this study, I explore 
perspective taking in the lens of cognition and examine individual differences in its possible effect on 
stereotypicality and valence. 
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Perspective taking 


Perspective taking is the ability to put oneself in the place of others and recognize that other individuals 
may have points of view different from one’s own (Johnson, 1975). This is important since lacking such ability is 
identified as an obstacle that impairs mutual understanding among individuals with different identities and 
ideologies (Fitzpatrick et al., 2013). In the field of psychology, education, business, and law, perspective taking has 
been studied as an intervention that presumably affects out-group stereotypicality and valence (Galinsky and 
Moskowitz, 2000). Mostly, participants of such studies were given a photograph of an out-group individual and 
instructed to adopt the perspective of the individual; more specifically, they were asked to “imagine a day in the life 
of the individual as if they were that person, looking at the world through his/her eyes and walking through the 
world in his/her shoes (Skarinko and Sinclair, 2013, p.12).” Contradicting reports, however, have been made 
regarding the effect of such intervention. Some argued that perspective taking is one of the best strategies to reduce 
stereotypicality as well as valence (Galinsky and Moskowitz, 2000; Davis et al., 1996) while others contended that it 
has little to a negative influence on stereotypicality and valence (Skorinko and Sinclair, 2013). 

Stereotypicality 

Stereotypicality may be defined as the degree of stereotype to which individuals may hold on the target. 
Two methods that measure stereotypicality introduced by Galinsky and Moskowitz (2000) are rating stereotypicality 
shown in narrative essays and lexical decision task. On one hand, participants were asked to write a narrative essay 
about the individual shown in the photograph as they perspective take. Then, raters estimate the overall 
stereotypicality of the contents of participants’ narrative essays, using a 9-point scale for the judgments with one 
scale anchored at 1 (not at all stereotypical) and 9 (very stereotypiclal). On the other hand, participants were given 
10 words (5 of which were stereotype-consistent and 5 of which were stereotype-irrelevant) through computer 
screens. Among those 10 words, one word is given each time and participants were asked to decide and press either 
key (one labeled as word and one labeled as non-word). Their response time were recorded and analyzed as to 
decide the degree of stereotypicality. As discussed earlier, results of the effect of perspective taking on 
stereotypicality are contradictory. Some argued that perspective taking reduces stereotypicality by encouraging 
thoughts towards the out-group to be more “selflike” (Galinsky and Moskowitz, 2000; Davis et al., 1996); while 
others asserted that it could rather enhance stereotypicality by people using stereotypes as a basis for perspective 
taking (Skorinko and Sinclair, 2013). 


Valence 

Valence may be defined as the degree of positive or negative attitude to which individuals may hold on the 
target. Two methods that measure valence are rating valence shown in narrative essays (Galinsky and Moskowitz, 
2000; Davis; etc.) and Implicit Bias Test (Nosek; Banaji). On one hand, participants were asked to write a narrative 
essay about the individual shown in the photograph as they perspective take. Then, raters estimate the overall 
valence of the contents of participants’ narrative essays, using a 9-point scale for the judgments with one scale 
anchored at 1 (very negative) and 9 (very positive). On the other hand, participants were given IBT test and their 
ability to match positive and negative words with the target group is analyzed. Similar to stereotypicality, conflicting 
reports regarding results of the effect of perspective taking on valence have been made. 

Experiment 

The experiment was designed to investigate the rather contradicting reports about the effect of perspective 
taking on stereotypicality and valence. That is, in this study, I examined participants’ thought process associated 
with perspective taking and narrative essay task in order to generate hypotheses and ideas about perspective taking, 
stereotypicality and valence. 

Participants and design 

There was a total of four participants. Participants were two young(er) people, one male and one female, 
and two old(er) people, one male and one female, who were tested individually. Participants were selected for this 
study based on their age and gender (Participant 1 - female, age 27; Participant 2 - female age 61; Participant 3 - 
male, age 31; Participant 4 male, age 63). I conducted face-to-face interviews either in person or using Skype. Each 
interviews lasted less than an hour (approximately 35 to 55 minutes). 
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Procedure 

Participants were given instructions very similar to those given by Galinsky & Moskowitz (2000). To 
begin, all participants were shown black and white photographs (presented on a computer screen) of an older man 
and woman sitting on a chair. Since some argue that the effect of perspective taking on stereotypicality and valence 
may differ whether participants were shown a stereotype-consistent photo or not, 1 intentionally chose neutral 
pictures of elderlies in this study. 



Then, participants were instructed to adopt the perspective of the individuals in the photograph and 
“imagine a day in the life of these individuals as if you were that person, looking at the world through his/her eyes 
and walking through the world in his/her shoes.” Participants were then told to construct a narrative essay on a sheet 
of paper with 27 lines and told to take approximately 5 to 15 minutes to complete the task. Participants were asked 
to stop writing their narrative essay after 15 minutes had passed. 

Added to Galinsky & Moskowitz (2000) research, particularly, was the interview. First, participants were 
asked to provide general retrospective verbal reports about their essays. This took place right after participants 
completed the task to help participants generate verbal report with access to a specific memory trace of the episode 
(Ericsson, 1984). Specifically, participants were asked to “scan your narrative essay line by line and tell me what 
you thought about when you were writing those lines.” Then, I used prompts in order to examine the probable 
correlation between participants’ experience and their attitude towards the target. 1 paid more attention to 
stereotypical words (Table 1) as well as positive/negative words on participants’ paper (essays) and participants’ 
retrospective report as I probed. Their responses were audiotaped. 

Question 1: What did you notice from those pictures? 

Question 2: Is there anyone in the media (TV shows, news) that reminds you of those people? - How were 

older characters portrayed in the media? 

Question 3 : Is there anyone in your life that reminds you of those people? - How was the experience with 

him/her? 

Question 4: Do you remember anyone who talked about his or her experience with an older person? - What 
did they say? 

Question 5: Talk about your personality. - Are you usually empathetic? - If so, why do you think you have 
such tendency? 

Question 6: Do you usually think from others’ perspective? - If so, why do you think you have such 
tendency? 
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(Table 1 ) Representative Traits in the Stereotypes of Older People 


Stereotypes 

Traits 

Despondent 

neglected, sad, afraid, lonely 

Severely impaired 

feeble, slow thinking, senile 

Shrewd/ curmudgeon 

ill-tempered, complaining, prejudiced, stubborn, noisy 

Recluse 

timid, quiet, live in past, set in ways 

John Wayne conservative 

proud, patriotic, wealthy, conservative, religious 

Perfect Grandparent 

kind, generous, family oriented, wise 

Golden ager 

intelligent, productive, healthy, independent 


Observation 


Participant 1 (PI) 

The subject is a 27-year-old female. First, I told her to take approximately 5 to 15 minutes to complete the 
narrative essay task. However, during the task, she mentioned that she could not think of anything else and could not 
fill in all the lines. So, after 15 minutes, I asked her to stop composing her essay. She ended up filling in 19 lines. 
Next, I conducted an interview that took an additional 22 minutes. During the interview, she first explained her 
thought process while she was completing the narrative essay task. Then, she provided answers to my prompts. 

According to PI, she could not fill in all the lines because the picture was not detailed enough for her to 
come up with ideas. She mentioned that the old lady in the picture seemed to be looking at something but the picture 
gave no clues of what that would be. For this reason she had a hard time coming up with a story of the person in the 
picture. Even so, when I prompted her, PI answered that she thinks she is a good perspective taker. 

In her essay, PI identified the woman in the picture as an old person who lives in the U.S. but who came 
from Poland. For instance, on line 8 PI wrote ‘she lives in the U.S. but she came from Poland with her parents when 
she was 6.’ Similarly on line 10, PI wrote ‘they live in a neighborhood where other Polish-related families live.’ 
Throughout the interview, PI said she first thought of a day of the older person, then thought of a story behind that 
person. Since the person in the picture seemed to be dressed like a Polish women, PI made up a story that the person 
might have family members who are Polish and neighbors who are Polish as well. 

Also in her essay, P 1 assumed that the woman in the picture has a husband, a daughter, a son, and 
grandchildren. Besides, she suspected that the old lady in the picture seemed to have a life, which revolves around 
her family. For example, on line 1, PI wrote ‘wake up early because she has a household to run.’ Then, on line 2, PI 
wrote ‘[she and her husband] are retirees [,] but they are busy with their grandkids.’ In the interview portion, PI 
mentioned that her narrative essay might have been influenced by her grandmother, who has a good relationship 
with her grandchildren, including PI. 

Participant 2 (P2) 

The subject is a 61 -year-old female. I told her to take 15 minutes to complete the narrative essay task, so 
she took 15 minutes to fill in all 27 lines. Next, I conducted an interview that took an additional 36 minutes. During 
the interview, she first explained her thought process while she was completing the narrative essay task. Then, she 
provided answers to my prompts. 

Noticeably, in her narrative essay, she used words such as ‘lonely’ and ‘sick’ several times. For example, 
on her 1 1 th line she wrote ‘she called to her daughter and her daughter said she is sick due to a bad flue.’ On line 12, 
P2 wrote ‘she felt sad and lonely.’ Later, on line 26, P2 wrote ‘she felt lonely that she had no one to talk [to].’ 

During the interview, I asked P2 the reason she wrote ‘lonely’ and ‘sick’ several times; and P2 answered ‘it is [from] 
the picture.’ P2 mentioned that the woman in the picture looks lonely, unhappy, thin and weak. Moreover, later 
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during the interview, it turned out that some of her friends and her colleagues either are lonely and sick or have older 
people around them who are ill and hospitalized. 

During the interview, when I prompted P2 whether she would describe herself empathetic, she answered ‘I 
am not empathetic naturally.’ P2 described herself as an unemotional but logical and intellectual person. Yet she 
mentioned that as a Christian, her religion has taught and trained her to be empathetic. P2 explained that there are 
two great commandments in her religion. The second great commandment, ‘love your neighbors as you love 
yourself,’ particularly taught her to be considerate of others’ needs. She said she has tried to be obedient to this 
commandment and has practiced it for long. She thinks this led her to become a trained perspective taker. 

Participant 3 (P3) 

The subject is a 31 -year-old male. First, I told him to take 15 minutes to complete the narrative essay task. 
However, during the task, he mentioned that he could not think of anything else and could not fill in all the lines. So, 
after 15 minutes, I asked him to stop composing his essay. Like PI, P3 ended up filling in 19 lines. Next, I 
conducted an interview that took an additional 26 minutes. During the interview, he first explained his thought 
process while he was completing the narrative essay task. Then, he provided answers to my prompts. 

During the interview, P3 mentioned that he noticed a weathered wall in the picture. This led him to think 
that the park in the picture is located in a rural area. In fact, on line 5, P3 wrote ‘this is outside of downtown.’ P3 
added that since there are not many coffee shops or libraries around the rural area, the older person in the picture 
may not have many places where he can visit. For this reason, he mentioned in his essay several times that the old 
man in the picture would read books. For instance, on line 13 he wrote ‘I will stay here longer and read books’ and 
on line 18 he wrote ‘I read books or watch movies after dinner.’ Notably, P3 took the instruction, “imagine a day in 
the life of these individuals as if you were that person, looking at the world through his/her eyes and walking 
through the world in his/her shoes,” literally and used ‘I’ statement in his essay unlike other participants. 

P3 mentioned that he is a Christian during the interview, and he referenced this in his essay as well. For 
example, on line 6 he wrote ‘I often pray for my family here,’ and on line 19 he wrote ‘I pray before going to bed.’ 
Yet unlike P2, P3 did not relate his religion with him being empathetic. Rather, he thinks his experience of being a 
troubled teen influenced him to become a person who cares about others’ feeling. Moreover, he described himself as 
being a person who is not easily irritated, who is calm, and who likes to hang out with other people. He thinks this 
part of his personality has something to do with him being empathetic. In short, both P2 and P3 think that they are 
empathetic and good perspective takers, but for different reasons. 

Participant 4 (P4) 

The subject is a 63 -year-old male. First, I told him to take 15 minutes to complete the narrative essay task. 
However, during the task, he mentioned that he could not think of anything else and could not fill in all the lines. So, 
after 15 minutes, I asked him to stop composing his essay. P4 ended up filling in 12 lines. Next, I conducted an 
interview that took an additional 26 minutes. P4 asked to speak in Korean, so the interview was conducted in Korea. 
During the interview, he first explained his thought process while he was completing the narrative essay task. Then, 
he provided answeres to my prompts. 

Like P2, the 61-year-old female, P4 took a negative tone in his writing and speech towards the elderly 
person on the picture. For example, on his first line he wrote ‘ . . .woke up early in the morning and had nothing to do, 
lonely. . . ’ On his second line he wrote ‘ . . .no one to talk [to].’ Later in the interview, P4 mentioned that when he 
studied in the U.S., he saw elderly people exchanging small talk with grocery store clerks. This is, according to P4, 
one common characteristic of older people that originates from not having anyone around to talk to. Interestingly, as 
a Korean, he thinks his pessimism is affected by ‘Han.’ Han, a feeling of unsettled resentment, is a unique Korean 
cultural trait resulting from frequent foreign invasion (Kim, 2012). According to P4, it was the dominant cultural 
characteristic when he was young. 

During the interview, when I prompted P4 whether there is anyone in his life who reminds him of the older 
person in the picture, P4 answered ‘no.’ He said since he is teaching at a university, he rarely encounters elderly 
people. He also mentioned that at his age, it is not common to talk about such older people; so, he did not have 
opportunities to hear much about older people. In other words, he thinks that he and people in his social groups are 
not old enough to discuss a life of an elderly represented in the picture. 

Data analysis 

Although participants were given the same intervention, which was to imagine a day of the elderly in the 
picture as if the participants were those elderly, their essays and their thoughts behind writing those essays were all 
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different. Three out of four participants appeared to hold stereotypes of older people. PI (27-year-old female from 
Greece) described the older woman in the picture as a perfect grandparent whose life revolves around her family. 
Both P2 (61-year-old female from Korea) and P4 (63-year-old male from Korea) thought of the older person in the 
picture as being despondent and severely impaired. In addition, younger participants (PI and P3) seemed to have 
positive attitude towards the elderly while older participants (P2 and P4) seemed to have negative attitude towards 
the elderly. Table 2 summarizes these findings. Although the sample size, here, is too small to make major 
conclusion, it seems that individual differences in perspective taking may exist based on these findings. 


(Table 2) Participants’ Stereotypes of and Attitudes Toward Older People 


Participants 

Characteristics 

Stereotypes 

Attitude 

PI 

27-year-old female from Greece 

Perfect grandparent 

Positive 

P2 

61 -year-old female from Korea 

Despondent, Severely 
impaired 

Negative 

P3 

31 -year-old male from Korea 

- 

Positive 

P4 

63 -year-old male from Korea 

Despondent, Severely 
impaired 

Negative 


In explaining such individual differences, I first consulted Rumelhart’s framework of schema. According to 
Rumelhart (1980), people build up a vast repertoire of schemata through experience; existing schemata activate 
other schemata; and schemata account for perceiving, understanding, remembering, and learning. Since narrative 
essay task in this study mostly requires participants to perceive the pictures given, I scoped my analysis to include 
only the perceiving part among four major functions of schemata: perceiving, understanding, remembering, and 
learning. Then, I figured and analyzed certain patterns that may shape individual stereotypicality and valence: (1) 
individuals’ cultural experiences, (2) individuals’ social experiences, (3) and individual differences in perception. 

Cultural experiences 

Here, participants’ country-specific remarks were classified as cultural experiences. One example of the 
contribution of such cultural experiences to schemata (stereotypicality and/or valence) was that PI alone described 
the older woman in the picture as who lives in U.S. but is originally from Poland. During the interview PI attributed 
this thought to the picture, especially the way that the woman in the picture is dressed. 

PI (Retrospective Report): I am not sure where I got the idea of it [Polish woman]. I don’t really know 
myself a Polish older woman. . . The surrounding of the park just immediately made me think of the U.S., 
but she seems like a person that is not born and raised [in U.S.]. [This is because] I don’t think American 
grandmothers have that thing [on their head]. The way she [is] dressed reminds me of the place [Poland.] 

Since PI was the only participant who was born and raised in a European country, Greece, she appeared to 
have a schema that American grandmothers do not use hoods while European grandmothers do. Identification of the 
elderly in the photograph as a European would hardly occur among those who have no cultural experiences related 
to European countries. Thus, although ‘Polish women’ is not a stereotypical word, it is important to note the 
possibility of the effect of cultural experiences on perspective taking and stereotypicality. 

Another example of the influence of cultural experiences on schemata (stereotypicality and valence) was 
P4’s statement of Korea-specific trait, ‘Han.’ It comprises a mixture of feelings such as sadness, resentment, 
negativity, anger, etc. According to P4, this Han was a unique and dominant cultural trait of Korea when he was 
young. And his pessimism is greatly affected by such culture. 

Interviewer: [Translated] How would you describe your personality? 

P4: [Translated] I don’t consider myself as an optimist. The negative part of me is rather acquired than 
given. . . I didn’t grow up in a positive environment. My parents’ generation was one of the most 
unfortunate. They were the ones who experienced both Japan and Korean wars. They had to survive out of 
nowhere. . . I grew up being massively influenced by their resentment, which we call ‘Han.’ 
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Among 4 participants, P2 is suspected to have similar cultural experiences as P4 since they are both in their 
early 60s and both from Korea. Surprisingly, both P2 and P4 showed similar stereotypicality, despondent and 
severely impaired; and negative attitude toward the elderly. Though P2 did not mention the cultural affect, its 
correlation with stereotyplicality and attitude is a subject to investigate in the future. 

Social experience 

In this study, participants’ relationship-specific remarks were classified as social experiences. Based on the 
interview, it is suspected that P2’s frequent usage of the word ‘lonely’ and ‘sick’ may be related to her interaction 
with her friends and colleagues. 

Interviewer: Do you remember anyone who talked about his or her experience with an older person? 

P2: Yes, I have lots of friends who talked about their parents, especially, how lonely they are. . . I also have 

a colleague who retired many years ago. She did not marry so she lived alone. And now she is really sick, 

and there is no one to take care of her. So she feels very lonely and depressed these days. 

P2, peculiarly, had much accessibility to the elderly. She either heard about them through her friends or she 
had friends who were older people. Through her network of people she may have developed a repertoire of the 
elderly as sick and lonely. It is of a subject for further examination, the impact of perspective taking on 
stereotypicality and attitude with people like P2 who already have strong stereotyplicality. 

Unlike P2, P4 had little access to the elderly in his daily lives. Interestingly, P4 seemed to suffer the most 
among all the participants in his narrative essay task. For 15 minutes given to complete the narrative essay task, P4 
could only fill in 12 lines, the least among all the participants. During the interview, P4 mentioned that he and 
people in his social groups are not old enough to discuss about a life of an elderly represented in the picture. Thus, it 
seems that P4’s inability to fill in the lines in his narrative essay task may be related to his lack of accessibility to the 
elderly in the picture. 

Interviewer: Do you remember anyone who talked about his or her experience with an older person? 

P4: [Translated] No, since I am teaching at a university I rarely meet older people. People I mostly 

encountered with are young college students. Also, at my age, it is not common to talk about the elderly. 

My friends and I rarely talk about such older people. 

Perception 

The most salient figure among the components of the pictures given were ‘hood’ on the head of the elderly 
woman for PI, ‘thin leg’ of the elderly woman for P2, ‘old wall’ in the background of the picture for P3, and 
‘leafless tree’ next to the elderly man for P4. Some seem to be connected to stereotyplicality and valence. For 
example, leafless tree may seem to relate to P4’s stereotypicality (despondent) and negative attitude toward the 
elderly. Nevertheless, what is important here is that participants perceived differently even if they looked at the same 
picture. Skorinko and Sinclair (2013) asserted that different images (stereotype-neutral and stereotype-enhancing 
pictures) might affect the effect of perspective taking on stereotypicality and valence. This study, however, shows 
that individual differences in perception, not differences in images, may affect relation between perspective taking 
and stereotypes/attitude. 


( Table 3) Participants’ Major Perception 


Participants 

Perception 

Stereotypes 

Attitude 

PI 

Hood 

Perfect grandparent 

Positive 

P2 

Thin Leg 

Despondent, Severely 
impaired 

Negative 

P3 

Old Wall 

- 

Positive 

P4 

Leafless Tree 

Despondent, Severely 
impaired 

Negative 


143 




Discussion 


First of all, it is important to note that the effect of perspective taking on schemata (e.g. stereotypicality 
and/or valence) may vary by individual differences in social and cultural experiences as well as perception. Skorinko 
and Sinclair (2013) contended that perspective taking could enhance stereotypicality by people using stereotypes as 
a basis for perspective taking. Individuals may hold strong stereotypes or negative attitude towards the others 
depending on their previous social and/or cultural experiences. When these stereotypes are used as a starting line to 
take the perspective of others, perspective taking may produce some unintended negative consequences. 

Accordingly, careful examination of the effect of perspective taking is required before the actual 
application of such activities in the field. Youniss et al. (2002) maintained that schools in the future will play a critical 
role in helping incorporate increasingly diverse populations into societies. Schools can help collaboration among different 
groups by instituting civic curricula that enhance understanding, tolerance, and respect for differences (Bransford et al., 2000). 
Existing perspective-taking products such as Political Agenda (Easterday et al., n.d.) and Fair Play aims at realizing such 
enhancement. However, the lack of research on contradicting reports of the effect of perspective taking makes the 
stakes of using such civic products incredibly high. 

Thus, future studies should include the investigation of individual differences in perspective taking and its 
effect on stereotypicality and valence. Although, this study shows possibilities of the existence of individual 
differences in people taking perspective of others, sample size and test methods were somewhat limited to make a 
valid supposition. Hence, in the future, larger sample size and objective tests -Lexical Decision Test (Galinsky and 
Moskowitz, 2000) to test stereotypicality and Implicit Bias Test (Banaji and Greenwald,2013) to test valence-are 
required for concrete conclusion of this report. 
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Abstract 

Despite the growing body of research on the effects of different strategies on cognitive presence in online 
discussions, there is little known about the design of question prompts that may influence cognitive presence in 
online discussions. In this study we examined the influence of question prompts designed with the Practical Inquiry 
Model (PIM), compared to the regular (playground) questions, on students’ levels of cognitive presence in online 
discussions. Students’ discussion postings were collected and categorized according to the four levels of cognitive 
presence: triggering events, exploration, integration, and resolution. The data were analyzed using quantitative 
content analysis, descriptive statistics, and non-parametric statistics. Results revealed that students’ responses to 
question prompts based on the PIM resulted in higher levels of students’ cognitive presence-integration of ideas and 
resolution of problems-compared to the responses based on the regular (playground) questions. These results 
suggest that instructors can use the PIM both as an analysis tool and a guiding framework to design questions that 
may influence cognitive presence in online discussions. 

Introduction 

The importance of cognitive presence in online learning environments has been widely reported in 
literature (Garrison, Anderson, & Archer, 2001; Rourke & Kanuka 2009). Cognitive presence is associated with ‘the 
extent to which learners are able to construct and confirm meaning through sustained reflection and discourse in a 
critical community of inquiry’ (Garrison et al., 2001, p. 11). The concept of cognitive presence emerged from the 
Community of Inquiry (Col) framework proposed by Garrison, Anderson, and Archer (2000), to guide the use of 
online learning environments in support of social constructivist approach to learning. 

Col is grounded in the Practical Inquiry Model (PIM), which describes four levels of cognitive presence 
that can be observed in students’ online discussions postings: (1) Triggering- becoming aware of a problem through 
initiating the inquiry process; (2) Exploration- exploring a problem by searching for relevant information and 
offering explanation; (3) Integration- interpretations/construction of possible solution, and (4) Resolution- applying 
or defending potential solutions with a new thought/idea. Triggering and exploration phases represent the low-levels 
of cognitive presence where students are presented with a triggering question and then they begin to understand the 
problem through engaging in a meaningful dialogue. On the other hand, the integration and resolution phases 
represent high-levels of cognitive presence where students build on each other’s ideas and synthesize information to 
provide real world solutions. 

These four phases are not considered linear since students may return to a previous phase (Swan, Garrison, 
& Richardson, 2009). Instructors may ask questions to help move the conversation by fostering deeper levels of 
thinking, reflection, and critical inquiry among the four stages. According to Garrison et al. (2001), the PIM focuses 
on thinking processes as opposed to individual learning outcomes. Therefore, it can be used as a tool to assess 
critical discourse and higher-order thinking in online discussions. According to Schrire (2004), the PIM is “most 
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relevant to the analysis of the cognitive dimension and represents a clear picture of the knowledge building 
processes occurring in online discussions” (p. 491). 

Although, achieving high levels of cognitive presence is often the goal of online discussions, some studies 
have noted that majority of students’ discussion posts reflected lower levels of cognitive presence, i.e., surface 
exploration of the course content (Rourke & Kanuka, 2009). McLoughlin and Mynard’s (2009) found that low 
levels of cognitive presence in online discussions were due to the nature of the task and the wording of the question 
prompts. 

Carefully designed questions are required tools to cognitively engage students and facilitate knowledge 
construction (Chin, 2004; Wang, 2005). Although, there are many purposes of questioning (e.g., promoting recall of 
information, encouraging reflection, and focusing attention), stimulating higher order thinking is widely 
acknowledged in the literature (Blanchette, 2007; Gibson 2009). Several studies have used six out of nine question 
typology proposed by Andrews (1980) including brainstorm, general invitation, funnel, playground, lower- 
divergent, and analytical convergent to explore the relationship between initial discussion question and levels of 
critical thinking (Bradley, Thom, Hayes & Hay, 2008; Ertmer, Sadaf, & Ertmer, 2011; Richardson, Sadaf, & Ertmer, 
2012). Among these, the most frequently used question type was a regular or so-called “ playground ” (PG) prompts. 
The PG questions facilitate high level learning, as they require students to explore, analyze, and interpret a specific 
aspect of the course material or “playground” for discussion. 

Asking thoughtful questions plays an important role in inducing students’ higher-level cognitive processes 
(Blanchette, 2007; Ertmer et al., 2011; Richardson et al. 2012), such as reflection on practice, social negotiation, 
assimilation of information, and construction of meaning, all of which are integral to cognitive presence. When 
instructors ask questions that require integrating ideas and solving problems, students tend to critically examine the 
presented problems by the exchanging of viewpoints, exploring applications to problems, and synthesizing ideas to 
provide solutions, which is representative of the higher levels of cognitive presence (Hosier & Arend, 2012). In 
order to facilitate high level learning, instructors may consider designing questions that demand cognitive 
collaboration of learners to integrate, synthesize, and evaluate discussion ideas, resulting in meaningful discourse 
requiring cognitive presence. 

This study examined the influence of question prompts designed with the PIM on students’ levels of 
cognitive presence in online discussions. Only a few studies have been conducted in this area despite the established 
importance of initial question prompts to influence cognitive presence. For example, Richardson et al. (2012) found 
that a large number of student responses reached integration and resolution levels of cognitive presence when 
asking students to create solutions to authentic problems. On the other hand, Darabi, Arrastia, Nelson, Cornille, & 
Liang (2011) found that student posts mainly remained in the exploration stage even when students were asked 
questions to help them move to the resolution level. Darabi et al. (2011) concluded, “Discussion strategies should be 
designed to promote progression through the phases of cognitive presence with the intention of contributing to 
higher-level learning” (p. 225). In this sense, the PIM may provide an effective framework for designing questions 
that facilitate students’ cognitive presence beginning with lower levels of recognizing the problem and progressing 
to higher levels of solving the problem and creating a solution. 

Purpose of the study 

The purpose of this study was to examine whether question prompts based on the PIM led to students’ high 
cognitive presence in online discussions. The research questions included: 

1. To what extend question prompts based on the PIM, compared to the PG question prompts, lead to the 
highest levels of cognitive presence — integration and resolution — in online discussions? 

2. Is there a difference in the overall cognitive presence facilitated by prompts based on the PIM and the 
PG questions? 

3. Is there a difference in the levels cognitive presence (triggering, exploration, integration, and 
resolution) in the discussions facilitated by prompts based on the PIM and the PG questions? 

Methods 

This study used a mixed methods research design including quantitative content analysis, descriptive, and 
non-parametric statistics. Qualitative data were collected from the online discussions and transformed into 
quantitative data (frequencies) through coding. Comparisons were made between the levels of cognitive presence 
generated by question prompts based on the PIM and the PG question prompts using descriptive and non-parametric 
statistics. 
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Participants 

Twenty-four students (9 males and 15 females) participated in this study. All the students were enrolled in 
an online masters program in Curriculum and Educational Technology. The online course was offered over a 16- 
week long semester and delivered via a learning management system. Blackboard. The students ranged in age from 
21-45 years. Most (w=19) of the participants had taken three or more online courses prior to participating in this 
study. 

Context of the Study 

Students were required to engage in online discussions as part of their course grade. During the semester, 
there were eight discussion prompts on various topics on instructional design and technology. The course was 
created such that students participated in four instructor-designed and facilitated discussions during the first half of 
the course, followed by facilitation of two to three student-designed and facilitated discussions during the second 
half of the semester. For this research, the first four online discussions designed by the instructor were used. Of the 
four discussion question prompts, two were designed based on the PIM (PIM#1 and PIM#2) and two were the PG 
question prompts (PG #1 and PG#2). Below is a diagram showing the order of question prompts provided in all four 
online discussions and facilitated by the course instructor, who is the primary researcher of this study, during the 
first four weeks of the course in the fall 2014. 


Week 1: PG#1 ■ ^Week2:PIM#l ■ ^Week3:PIM#2 ■ ^Week4:PG#2 


• Educational 


•Learning 


•Constructivist 


•Evaluation in 

Technology 


Theories 


Learning 


Instructional 

Definition 




Theory 


Design 

V J 








Figure 1. Timeline of four online discussions at the beginning of the course 


The PIM question prompts 

For this type of question prompts, students were presented a case followed by four discussion question 
prompts representing four levels of cognitive presence, designed to advance students through the phases of cognitive 
presence in the context of the case -based discussions. Students were required to post responses to two different 
discussion threads. The first discussion thread required response to the first two question prompts (triggering and 
exploration) and one comment on other students’ post by midweek. The purpose of asking triggering and 
exploration question prompts during the first half of the week was to first help students understand the nature of the 
problem and then search for relevant information to provide possible explanations. 

Example: What do you think are the problems with the way Mr. Evans has designed his instruction 
[ Triggering question prompt ]? Flow can your [use the one you have been assigned] theoretical perspective 
help to understand the problems presented in this case [Exploration question prompt ]? 

During the second half of the week, students were required to post in the second discussion thread, which 
required one response to two question prompts (integration and resolution) and one comment on other student’s 
post. The purpose of asking integration and resolution question prompts was to help students create solutions based 
on their discussion and provide justifications for their solutions. 

Example: Briefly identify a key principle (or principles) taken from the theoretical perspective and explain 
how it would be applied to solve the learning problem presented in the case [integration question prompt ]. 
Justify your response by providing applications of your solutions in real world situations [resolution 
question prompt ]. 


The PG question prompts 

The PG question prompts ask students to explore a specific aspect of the course material or a concept and 
then apply the knowledge to provide the response (Andrews, 1980). In this study, the PG question prompts referred 
to a specific concept, such as a learning theory, and asked students for its analysis and application of the concept. 

Example: Using one or more of the evaluation models explained in the chapter [course concept ], explain 
how would you evaluate your final instructional design project [application of concept ]. 
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Data Collection and Analysis 


The data were collected from online discussion postings of students enrolled in EDTE 660 course taught in 
fall 2014. The course was offered through Blackboard Learn, which is a password-protected site. As the data are 
identifiable, the authors may share only anonymous scripts of students’ postings upon requests. 

Content analysis of four week long online discussions were conducted using Garrison et al.’s (2000) 
suggested analysis procedure in order to examine students’ levels of cognitive presence. The procedure involved: (1) 
segmenting conference transcripts into meaningful units, (2) classifying the units into one of the four phases of 
cognitive presence and (3) summing the frequency of units in each phase. Out of total 599 coded segments, 355 
segments were in response to the PIM question prompts and 244 were in response to the PG question prompts. Most 
posting and responses were given one code. If there was evidence of two types of cognitive presence within one 
posting or response, then two codes were given. For example, students’ postings or responses to integration and 
resolution question prompts in one discussion submission were coded as two segments. 

The data were coded by the primary researcher of this study and two research assistants using the cognitive 
presence coding scheme developed by Garrison et al. (2001). Before beginning coding, the primary researcher met 
with two research assistants to discuss and find agreement across the coding scheme and the definition of categories 
of cognitive presence. The transcripts were then coded independently while reviewing the codes and controlling for 
simple errors, and thereby increasing reliability. After extensive discussions, consensus was reached and frequencies 
in each category were counted. The transformed quantitative data were then analyzed with non-parametric 
statistical tests because of small non-random sample size («=24). The non-parametric Friedman Test was run to 
examine differences across all four online discussions. Once differences across all four discussions were identified, 
we applied the non-parametric Wilcoxon signed-rank test to compare differences between each online discussion to 
reveal the influence of each type of question prompts on each level of cognitive presence. 

Results and Discussion 

Overall, the results revealed that students posted more responses to the PIM question prompts («=355) 
compared to the PG question prompts (n=244) in each of the four online discussion per student (see Table 1). 

Table 1 


Average number of segments per student (n=24) response to question prompts 



PIM Question Prompts 

PG Question Prompts 


PIM# 1 

PIM #2 

PG# 1 

PG # 2 

Number of segments 

153 

202 

100 

144 

Average segments per student 

6.4 

8.4 

4.1 

6.0 


Research Question 1 

The first research question examined the extent to which the PIM question prompts, compared to the PG 
question prompts, led to the highest levels of students’ cognitive presence in online discussions. The comparison 
between the two types of question prompts revealed differences in percentage of segments within each level (see 
Table 2). The PIM question prompts resulted in higher percentage of segments within the integration (34%) and 
resolution (7%) level, as compared to the PG question prompts. Looking at the individual students’ learning 
outcomes, more than half of the students’ (15 of 24) responses reached the resolution level in the two discussions 
based on the PIM question prompts, whereas, only two students’ responses achieved the resolution level in 
discussions based on the PG question prompts. 
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Table 2 


Average number of student responses per question prompt (n=24) 


Cognitive Presence 


PIM Prompts 



PG Prompts 


PIM#1 

n (%) 

PIM#2 

n (%) 

Total 

n (%) 

PG#1 

n (%) 

PG#2 

n (%) 

Total 

n (%) 

Triggering event 

18(12) 

37(18) 

55(15) 

13(13) 

31(22) 

44(18) 

Exploration 

75(49) 

81(40) 

156(44) 

69(69) 

72(50) 

141(59) 

Integration 

53(34) 

69(34) 

122(34) 

16(16) 

41(28) 

57(22) 

Resolution 

7(5) 

15(8) 

22(7) 

2(2) 

0(0) 

2(1) 

Total 

153(100) 

202 (100) 

355(100) 

100(100) 

144(100) 

244(100) 


Research Question 2 

The second research question examined whether there were differences in the overall cognitive presence 
facilitated by the PIM and the PG question prompts. A non-parametric Friedman test showed a significant 
difference in the overall levels of cognitive presence across all four discussions, f (3, n= 24) = 33.04, pm .000. 
Figure 2 presents the median values change on levels of cognitive presence. There was an increase in the levels of 
cognitive presence from week 1 of PG#1 f.V/t/ — 4.00) to week 2 of PIM#1 (Md= 5.00) and to week 3 of PIM#2 
(Mi=9.00). However, levels of cognitive presence decreased in week 4 of PG#2 (Md= 5.00) showing that the PIM 
question prompts helped increase students’ level of cognitive presence in the discussions compared to the PG 
question prompts. The post-hoc analysis with the Wilcoxon signed-rank test with a Bonferroni correction indicated 
that student responses to the PIM question prompts resulted in significantly higher levels of cognitive presence than 
the PG question prompts. 



Week 1 (PG#1) Week 2 (PIM#1) Week3(PIM#2) Week4(PG#2) 

Figure 2. Median values change in quality of posting by the type of questions 

Further analysis with a non-parametric Wilcoxon signed-ranks test was conducted to reveal differences 
between each discussion. From week 1 of PG#1 to week 2 of PIM#1, the test indicated that levels of cognitive 
presence in response to the PIM question prompts was significantly higher than responses to the PG question 
prompts (z=-3.45,/?=0.001, large effect size r=.50). From week 2 of PIM#1 to week 3 of PIM#2, there was 
statistically significant increase in the levels of cognitive presence in responses to the two discussions (z=-2.78, 

/j — 0.005, medium effect size r=. 40). However, there was a statistically significant reduction in levels of students’ 
cognitive presence from week 3 of PIM#2 to week 4 of PG#2 (z=-3.454,j9=0.001, large effect size r=.50). 
Responses to the PIM question prompts resulted in higher levels of students’ cognitive presence compared to those 
based on the PG question prompts. 

Research Question 3 

The third research question examined whether there was a difference in the levels of cognitive presence 
(triggering, exploration, integration, and resolution) in the discussion facilitated by the PIM and the PG question 
prompts. Our examination revealed significant differences between discussions by levels of cognitive presence 
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including triggering, exploration, integration and resolution. A non-parametric Wilcoxon signed-rank test showed 
an increase at the integration and resolution levels with the PIM question prompts (see Figure 3). The median values 
at integration were higher in week 2 of PIM#l(Mi=2.00) and week 3 of PIM#2 (Md= 3.00) compared to week 1 of 
PG#1 (MaMl.00) and week 4 of PG#2 (Md= 1.00). The median value at resolution was found only in week 3 of 
PIM#2 showing that the PIM question prompts can move students’ responses to higher levels of cognitive presence 
compared to the PG question prompts. 

Further examination was conducted to identify the differences between the four levels of cognitive 
presence across all four online discussions (see Figure 3). A non-parametric Wilcoxon signed-rank test revealed a 
significant difference at the integration (z=- 3.52, p < .001, large effect size r=,51) from week 1 of PG#1 to week 2 
of PIM#1. From week 2 of PIM#1 and week 3 of PIM#2, there was a significant increase at the triggering (z=-2.71, 
p =.007, medium effect size r= .39); integration (z=-2 . 1 2, p = .034, medium effect size r=.31) and resolution (z=- 
2.33, p=.02, medium effect size r=,34). Flowever, from week 3 of PIM#2 and week 4 of PG#2, there was a 
significant reduction at the exploration (z=-2.05,/>=.04, medium effect size r=.29), integration (z=-3.07, jt>=.002, 
medium effect size r=.44) and resolution (z=-3.6 1 , /;=.()()(), large effect size r=. 52). This showed that students’ 
levels of cognitive presence (exploration, integration, and resolution) in discussions were high when they responded 
to the PIM question prompts compared to the PG question prompts. However, the increase in triggering, integration, 
and resolution levels between PIM#1 and PIM#2 needs to be me more closely examined. 
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Figure 3. Median values change in quality of posting by the levels of cognitive presence 

These results support the notion that discussions can reach high levels of cognitive presence, progressing to 
integration and resolution phases, when instructors ask questions that require students to provide a solution or lead a 
discussion to a meaningful resolution of ideas (Hosier & Arend, 2012; Richardson et ah, 2012). This emphasizes the 
importance of discussion question types that requires students to move from their initial understanding levels to 
those that require students to critically assess the problem, create a solution as well as justify the solution offered. 
However, these findings diverge from Darabi et al. (2011) results that suggested that even though students answered 
the resolution questions, they did not reach high levels of cognitive presence due to lack of deep interaction. One 
possible explanation is the way the question prompts were designed or the interaction that was required. 

McLoughlin and Mynard (2009) found that the reason for the low cognitive levels was due to the nature of the task 
and the wording of the prompt. In our study, the resolution question prompts were worded explicitly to ask students 
to provide solutions as well as justify their solutions with examples of real world applications. In addition, students 
were given an authentic task of providing a solution to a case that was authentic and meaningful to their learning. 
The students were also required to post meaningful comments and were encouraged to ask questions to understand 
their peers’ solutions. 



Conclusion and Implications 

Designing online discussion questions based on the PIM can influence students’ cognitive presence in 
online discussions. The PIM not only provides a useful means for understanding cognitive presence in online 
discussions, but can also serve as a guiding framework to design questions. Looking at the results, one may conclude 
that students can reach high levels of cognitive presence when discussions are designed with the PIM questions 
prompts accompanied with well-designed tasks and participation guidelines. 


151 


As previous studies found out that the majority of students’ postings in online discussions usually remain at 
the lower level thinking (Bradley et ah, 2008; Rourke & Kanuka, 2009), online instructors can modify their 
instructional methods and ask questions that require students to move towards higher levels of cognitive thinking. 
Attempting to move discussion through the stages of cognitive presence — triggering, exploration, integration, and 
resolution — may lead to high levels of learning as each stage offers a process which encourages knowledge 
construction through deep levels of thinking and critical discourse. Question prompts that explicitly ask students to 
provide a rationale for their solutions can provide students a way to critically think about their learning and step 
back to examine their own solutions (Hosier & Arend, 2012). In addition, it is important to give students an 
authentic task such as a case or a problem to solve followed by PIM prompts can make discussions relevant to their 
learning. Furthermore, framing the entire activity to guide the process of student engagement and interaction may 
also be helpful. Finally, students can be given one week for each type of question to provide more time for students 
to reflect on triggering/ exploration before responding to the integration/resolution questions. In this way, a high 
level of thinking is driven by the instructor's questioning technique. Further research investigating the relationship 
between the levels of the PIM question prompts and the levels of cognitive presence in students’ postings and the 
discourse facilitated in follow-up responses would be promising directions for future studies. In addition, the 
increase in the cognitive presence levels between PIM prompts needs to be me more closely examined. 
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Abstract 

The purpose of this study was to analyze current conversations about trending educational technology 
topics across scholarly research, trade journals, and social media. Doctoral students in a Trends & Issues in 
Educational Technology’ course selected trending educational technology topics to thoroughly analyze conversations 
through a systematic protocol. Across the selected topics, we found differences in conversations across scholarly 
journals, trade publications, and social media. We did not find similarity in authorship across media, but we did 
discover general publication patterns across media. The process followed to analyze theses conversations across 
scholarly journals, trade publications, and social media was determined to be extremely useful. 

Introduction 

Conversations about educational technology happen in a variety of settings — from the peer-reviewed pages 
of a scholarly journal, to trade publications for practitioners, to social media feeds. Researchers have analyzed the 
educational technology conversation in scholarly journals by topic (Giannakos 2013, Hsu & Ching 2013) and by 
journal (West & Bomp 2014). However, it is also important to know what is being talked about and who is 
participating in less formal conversations in trade journals and social media (Martin, et al. 2011; Sullivan, et al. 
2012). To effectively bridge research and practice, it is critical to examine these conversations across media. The 
purpose of this study was to analyze current conversations about selected educational technology topics across 
scholarly research, trade journals, and social media. Each participating researcher selected an educational 
technology topic in his/her research agenda: 

• Use of asynchronous and synchronous learning 

• Barriers to blended learning 

• Educational technology professional development 

• Gamification 

• Play pedagogy 

• Technology and physical education 

• Technology and self-regulated learning 

• Technology integration 


Research Journal Analysis 

The Social Sciences Citation Index (SSCI) contains data from “3,000 of the world’s leading social sciences 
journals across 50 disciplines” ( http://thomsomeuters.com ). We used the SSCI to answer three key questions: What 
is the status of the conversation? Who is participating in the conversation? What is the conversation about? For each 
individual topic, we used advanced search strategies and limiters to gather data on the occurrence rate of research 
papers; article citation statistics; which subject domains and journals published research; author and 
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institution/organization publication frequency; types of research methods used; populations studied; and research 
questions, findings, and purposes. 

The initial search was limited to refereed journal articles and conference proceedings from 2004-2014. 
From a citation report for relevant returns, we used items published and citations per year to determine whether the 
conversation on each topic was increasing or decreasing. The analyze results tool provided data on authors, 
document types, languages, countries, funding agencies, organizations, publication years, research areas, source 
titles, and Web of Science categories. This data informed the research question of who (authors, 
universities/organizations, geographic locations, and disciplines) is participating in the conversation. 

The next step was to export the results to EndNote for more in-depth analysis of the content. To determine 
the types of research methods used in technology integration studies, we used the framework established in West 
and Borup 2014: 

• Descriptive research — papers employing an analysis of descriptive statistics (eg, means, standard 
deviations, percentages), often through the use of a survey instalment. 

• Inferential — papers employing inferential statistical analysis methods, including experimental, quasi- 
experimental, correlational and instalment development research designs. 

• Interpretative — papers employing qualitative methods to inductively interpret and develop findings from 
data, including data derived through interviews and observations. 

• Theoretical/philosophical — non-data based papers, such as literature reviews and discussions of emerging 
theoretical or design frameworks. This category was included to accommodate papers that did not report 
research, but provided value (and had been published by the journal) through other kinds of contributions. 

• Content/discourse analysis — descriptive analysis of conversation transcripts, online discussion transcripts 
or other data that could be coded according to a priori categories and then reported through descriptive 
means, averages and other descriptive statistics. This category was included to accommodate papers that 
reported statistics in a non-inferential way but for which the coding mechanism was not as interpretative as 
typical qualitative methods. 

• Combined or mixed methods — papers integrating two approaches (typically inferential and interpretative 
approaches) to address research questions best served by dual methodologies, (p. 547-548) 

A review of the abstract and/or the full article yielded information on what subject groups were studied and what 
research questions/problems were most often pursued. 

By copying all abstracts collected into WriteWords ( http://writewords.org.uk/phrase count. asp ), we were 
able to determine which two- and three-word phrases were most often used within each conversation. 

Trade Publication Analysis 

The ProQuest Research Library’ is a multidisciplinary database that indexes more than 6,000 titles 
( http://proquest.com ). Using the same search terms as we did in the scholarly journal search, we conducted a 
ProQuest search limited to trade journals only published from 2004 to 2014. Exporting the results to a spreadsheet 
allowed us to sort data according to title; abstract; authors; document types; publication details; and subject 
classifications, headings, and terms. This helped us determine whether the conversation regarding each topic was 
increasing or decreasing and who is participating in the conversation (authors, universities/organizations, geographic 
locations, and disciplines). The content of the conversation within trade publications was determined by copying all 
key word terms into WriteWords ( http://writewords.org.uk/phrase count.asp ) and noting which two- and three- word 
phrases were most often used. 


Social Media Analysis 

Since this research project was part of a semester-long course and most common social media tools were 
not available throughout our entire targeted timeframe (2004-2014), we decided to limit our examination of 
conversations in social media to Twitter for a selected timeframe within the bounds of the course. We chose to use 
Follow the Hashtag ( http://followthehashtag.com ) as a Twitter analysis tool because of the thoroughness of the data 
it collects, the ease of use, and the company’s offer of free premium access to student researchers in a course. Follw 
the Flashing provides the following data for a specified search term during a specified timeframe: 
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• Overview - total tweets, contributors, frequency, total impressions, gender, geo location, top 10 users by 
keywords and retweets, etc. 

• Influence - contributor follower statistics, aggregated influence score evolution, top contributors by 
influence, etc. 

• Content - top 10 influences by influence score, original tweets, retweets, replies, best hour of the day and 
day of the week, top 10 words, top 10 hashtag, embedded links, top images, etc. 

The occurrence rate and impressions of tweets served as data for the status of the conversation, the top users and 
influences answered the question of who is participating in the conversation, and top 10 words and hashtags 
informed the content of the conversation. 


Findings 

While findings were unique for each of the selected topics, we did identify general trends across all topics. 
First, the highest publication rates in scholarly journals were approximately two years later than trade publications. 
Second, the other disciplines engaged in conversation similar to educational technology are medicine and computer 
science. Finally, the top authors in one media are not top in other media. For example, none of the top authors in 
scholarly journals were also represented in the top lists for trade journals or Twitter. The most prolific trade journal 
author was “Anonymous,” and the top tweeters were often organizations or commercial enterprises. 

Use of Asynchr onous and Synchronous Learning 

The conversation on the use of asynchronous and synchronous learning revealed has steadily increased 
from 2004 to 2014, with the highest publication productivity from 2012 to 2014 for scholarly journals and 2008 for 
trade publications. The top research journals were Computers & Education, the International Review of Research in 
Open & Distance Learning, and the British Journal of Educational Technology >. Top trade publications were 
Education Business Weekly and Training & Development. The most prevalent contributions in all three media are 
from the United States and Canada. The primary fields participating in this conversation are education, medicine, 
computer science, and training. The conversation is largely focused on effectiveness, with research students on 
student achievement, learning outcomes, instructional design, and engagement, and trade publications focused on e- 
learning, training, practices, and tools. In conclusion, the conversation around blending asynchronous/synchronous 
tools in online learning is growing. As technology develops and the use grows, the frequency and depth of 
conversation will likely increase. 

Barriers to Blended Learning 

The status of the conversation on barriers to blended learning appeared to be cyclical as new technologies 
face challenges to adoption. The predominant fields participating in this conversation include nursing, dentistry, and 
higher education. There were very few search results for this specific topic - 171 journal articles (2004-2014), 67 
trade articles (2014-20014), and 7 tweets (April 21-27, 2015). One particular tweet, however, seemed quite 
appropriate: “When can we drop the modifier ‘blended’ and just set the expectation learning will include online 
components? #ncbls” (Greg Garner @classroom_tech). Perhaps this quote signals a change to come in the future of 
the discussion of barriers to blended learning in research studies and trade publications. 

Educational Technology Professional Development 

The conversation on educational technology professional development has experience a decrease since 
2013 in research journals and since 2012 in trade publications. The top research journals for this topic include 
Computers & Education, Australasian Journal of Educational Technology’, Educational Technology Society, the 
British Journal of Educational Technology, and the Turkish Journal of Educational Technology’. The top trade 
publications include T.H.E. Journal, Education Week, Technology’ & Learning, and Multimedia & 
Internet@Schools. The top five out of eight influences on Twitter were companies or organizations: Solution Tree, 
ISTE, NAESP, Remind, TeachFirst, and EducationDrive. 
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Gamification 


The conversation on gamification did not appear in scholarly journals until 2009 and has seen a dramatic 
increase in 2013. The same increase can be seen in trade publications in 2012. The conversation on this topic in 
Twitter is very active, with 10,243 tweets resulting in 36,227, 271 impressions over a two-week period in April 
2015. While the majority of research and trade articles are concentrated in the United States, Twitter shows active 
participation worldwide. The disciplines of computer science, psychology, information science/library science, 
education research, telecommunications, general internal medicine, engineering, communications, marketing, and 
computer and video games are all interested in gamification. The content of the research conversation is focused on 
the impact of gamification and gamification outcomes. 

Play Pedagogy 

Play pedagogy was the only topic of conversation that had decreased between 2004 and 2014 in both 
scholarly and trade publications. The discussion was largely focused on research and practice in early childhood or 
university education, but very few regarding junior high-high school levels. 

Technology and Physical Education 

The United States, the United Kingdom, Canada, and Australia are where conversations on this topic are 
taking place. Medicine, science, psychology, education, children, and physical fitness are the domains participating 
in the conversation. 

Technology and Self-Regulated Learning 

The search across media revealed a low level of conversation on this topic. There was a heavy emphasis on 
theoretical and interpretive research studies, and technology was not included in many of the discussions of self- 
regulated learning. 

Technology Integration 

The technology integration conversation largely centers on self-efficacy. The Twitter and trade publication 
conversations were largely driven by commercial entities or organizations. Top research journals on this topic 
include Computers in Education, Educational Technology’ Research & Development, and Educational Technology 
Society. Top trade publications include Technology’ & Learning, Computer Weekly News, and Education Business 
Weekly. It would be wise to repeat this analysis focused only on particular technology integration frameworks. 

Discussion 

The purpose of this study was to analyze current conversations about selected educational technology 
topics across scholarly research, trade journals, and social media. We sought to answer specific questions about 
selected topics: What is the status of the conversation? Who is participating in the conversation? What is the 
conversation about? We determined the process of analyzing scholarly journals with SSCI and WriteWords, trade 
publications with ProQuest and WriteWords, and Twitter with Follow the Hashtag was useful in gaining a broad 
overview and specific details about a topic. This is a process we all agree should be conducted at the beginning of 
any literature review and a process we plan to further refine and improve. 

First, we learned valuable lessons within the process undertaken. As we began analyzing research journal 
articles, we gained great appreciation for well-written abstracts containing the purpose, methods, results and 
conclusions of a study. We also determined that the “big picture” view gained from a search of SSCI to determine 
the status of the conversation, who is participating, and what the conversation is about should be undertaken at the 
beginning of any literature review and done periodically to be a good steward of that conversation. 

We learned there are a host of social media analytics tools available to work with. This is a rich and 
exciting way to delve into current ideas and trends on a topic. 

While we were able to answer our questions regarding specific topics across formal and informal 
conversations, we were left with additional questions. The highest publication rates in scholarly journals were one- 
to-two years behind trade publications. Does this necessarily mean that trade publications, largely written and 
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supported by commercial enterprises or organizations, are driving scholarly research? The other disciplines engaged 
in conversation similar to educational technology are medicine and computer science. Are we seeking collaboration 
opportunities with colleagues in these fields? Finally, the top contributors across the three types of media do not 
overlap. Should they? Should scholars seek to spread the message of their formal research into more informal 
conversations in trade publications and on Twitter? As scholars, we value conversation on the topics we are 
passionate about. We hope this process may help others analyze and engage fully in those conversations. 
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Abstract 

Educational technology is prevalent in higher education. An example of educational technology that has 
become ubiquitous at colleges and universities is a course management system (CMS). Although numerous benefits are 
associated with course management system use, both for students and faculty members, not all faculty members utilize 
a CMS. 

The purpose of this study was to determine the factors that influence CMS adoption for faculty members from 
community colleges in different Carnegie classification location settings. By determining the factors that most 
influence CMS adoption, college administrators and state leaders could initiate policy changes to attempt to influence 
all faculty members to adopt a CMS into their instructional practices. The study found that a reward structure was the 
factor that most faculty members acknowledged would be most effective in influencing faculty CMS adoption. Finally, 
several topics for future studies were recommended. 


Introduction 

It is evident that information technology, in general, is important in the United States and worldwide. 
According to a 2012 technology industry forecast, “Spending on IT software, equipment, and services is projected to 
reach $3.75 trillion worldwide this year, driven by solid growth in computer hardware and enterprise software” (Nagel, 
2012, p. 1). In higher education, an example of enterprise software that has become ubiquitous at colleges and 
universities worldwide is a course management system, such as Blackboard, Moodle, or Sakai (Jarrahi, 2010). The 
course management system (CMS) market in the U.S. has reached nearly $1 billion, and virtually every campus has 
such a system (DeFranco & Malm, 2011). CMSs have played an important role in higher education because they 
provide a web-based platform that allows colleges and universities to host and offer online, blended, and supplemental 
courses. CMSs typically offer a variety of tools that allow instructors to post documents, distribute grades, and 
collect/return assignments, as well as communicate with students via web-based email, discussion board, and 
synchronous chat (Gautreau, 201 1). Further data that underscore the importance of technology in higher education, 
specifically CMSs, can be found in a 2012 report from Babson Survey Group and the College Board that noted “Over 
6.7 million students were taking at least one online course during the fall 2011 term, an increase of 570,000 students 
over the previous year” (Sloan Consortium, 2012). Although the significant number of college students taking online 
courses reinforces the importance of technology in higher education, specifically, web-based course management 
systems, CMSs also support other types of courses in higher education. 

Research has indicated that there are a variety of different types of course formats that use a course 
management system. The focus of a great deal of research in this area focuses on purely online courses, which are 
defined as courses in which 80% or more of the content is delivered in a completely online format (Allen & Seaman, 
2010). Two additional course formats that students have encountered using a CMS include blended and web-facilitated, 
or supplemental. Blended courses have been defined at those that deliver 30 to 79% of course content online, and web- 
facilitated or supplemental courses have been defined as traditional courses that meet face to face in the classroom but 
also utilize a CMS (Allen & Seaman, 2010). 
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Purpose of the Study 


Instructional technologies, and specifically course managements systems, have become a staple at virtually 
all colleges and universities in the U.S. It is clear that community colleges in the U.S. educate a substantial portion 
of the overall number of college students; however, community college faculty members seem to utilize CMSs less 
frequently than their four-year university colleagues (Caruso, 2007). This seems to place community college 
students at a disadvantage because numerous benefits have been reported with CMS use, both for students and 
faculty. 

This study seeks to identify and understand the most important factors that influence faculty to adopt CMSs 
into their instructional practice. Although there are a number of different types of instructional experiences students 
and faculty can have with a CMS, the focus of this research study is not limited to faculty using a course 
management system for online instruction, but also for other types of instructional experiences, such as blended or 
supplemental. By identifying the most important CMS faculty adoption factors, community college and state leaders 
can gain a better understanding of their faculty regarding CMS adoption and initiate policy and programmatic 
changes that bolster CMS usage. Additionally, a substantial number of community colleges students have failed to 
realize the benefits of course management systems because their instructors have chosen not to use these systems. 

Of the approximately 7.7 million community college students, approximately 40% do not use a CMS (Caruso, 

2007). 

The findings of this study could influence more community college faculty to utilize a CMS and benefit 
their students. Course management systems have been used in a variety of different ways to create a variety of 
different learning experiences in higher education. The three primary types of CMS-facilitated learning experiences 
students’ encounter in higher education include (1) fully online courses, (2) blended or hybrid courses, and (3) 
supplemental or Web-facilitated courses. Online courses deliver content entirely online through a CMS and typically 
do not require face-to-face meetings; blended or hybrid courses integrate face-to-face instruction with Web-based 
instruction via a CMS; and Web-facilitated or courses meet regularly in the classroom but also utilize a CMS to 
augment the traditional face-to-face course (Allen & Seaman, 2010). Finally, examining faculty perceptions 
regarding CMS use will help provide a better understanding of the potential differences between the needs of full- 
time faculty and part-time faculty, as well as faculty from different location such as rural, suburban or urban 
settings. 


Methods 

Approximately one third of all college students (7.7 million) attend a community college (Carnegie 
Foundation for the Advancement of Teaching, 2010; Caruso, 2007). According to a 2007 report by the EDUCAUSE 
Center for Applied Research (ECAR), community college students indicated that only 49% of their courses used a 
CMS, while four-year university students reported that 66.7% of their courses used a CMS. Based on this report, it 
would seem crucial to gain a better understanding of the reasons why a substantial portion of the 400,000 faculty 
members teaching at community colleges (Kerste, 2011) have chosen not to utilize a course management system, 
especially since numerous benefits are associated with course management system use for faculty and students in 
online, blended and supplemental courses. Furthermore, limited research has been conducted that focuses 
specifically on community college faculty adoption of course management systems (West, Waddoups, Kennedy, & 
Graham, 2007). 

The intent of this study was to identify and understand the most important factors that influence faculty 
members to adopt CMSs. By identifying the most influential CMS faculty adoption factors, state and community 
college leaders, as well as faculty development professionals who wish to increase course management system 
adoption among faculty at their colleges could gain a better understanding of faculty perceptions regarding CMS 
adoption and initiate policy and programmatic changes that bolster CMS usage. 

Research Questions 

This study attempts to address the following research questions: 

1 . What factors most influence faculty members at two-year community colleges to adopt a course 

management system? 

2. What factors most inhibit faculty members at two-year community colleges from adopting a course 

management system? 
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3. Are the course management system adoption factors different for part-time faculty members and full-time 

faculty members at two-year community colleges? 

4. Are the course management system adoption factors different for rural, suburban and urban community 

college faculty members? 

The research design for this study is a non-experimental, descriptive survey design. According to Leedy 
(2005), “Survey research involves acquiring information about one or more groups of people — about their 
characteristics, opinions, attitudes, or previous experiences — by asking them questions and tabulating their answers. 
The ultimate goal is to learn about a large population by surveying a sample of that population; thus, we might call 
this approach a descriptive survey or normative survey”, (p. 183). Kerste (2010) noted that non-experimental 
approaches are superior for studying perceptions and beliefs. Further, survey research is designed to obtain data on 
specific variables from a sample of individuals that are representative of a group (Richy & Klein, 2007). The survey 
research design approach is well suited for the purpose of this study, which is to examine, using a web-based survey 
instrument, the factors that influence community college faculty members to adopt a course management system. 

Research Sites and Participants 

The research sites were chosen based on their Carnegie classifications. One institution had a Carnegie 
student enrollment classification of “medium,” which is defined as a community college with a student enrollment 
of 2,000-4,999, and the setting is considered “suburban” according to their Carnegie classification (Carnegie 
Foundation for the Advancement of Teaching, 2012). The second community college also is classified as “medium,” 
the setting classification is “rural” (Carnegie Foundation for the Advancement of Teaching, 2012). The third 
community college is considered “very large” based on its enrollment of over 10,000 students and is located in what 
is considered an “urban” setting based on their Carnegie classification (Carnegie Foundation for the Advancement of 
Teaching, 2012). Nationally, 43 community colleges share the Carnegie Classification of “medium” and “suburban,” 
113 share the Carnegie Classification of “medium” and “rural” and 42 are considered “very large” and 
“urban”(Carnegie Foundation for the Advancement of Teaching, 2012). The instructor population at the medium 
rural community college consists of 47 full-time faculty members and 151 part-time faculty members. The instructor 
population at the medium suburban community college consists of 60 full-time faculty members and 185 part-time 
faculty members. Finally, the instructor population at the very large, urban community college consists of 95 full- 
time faculty and 921 part-time instructors. All three Midwestern community colleges use Blackboard as their course 
management system. 


The Development of the Instrument 

A number of factors are associated with faculty members’ decision to adopt a course management system. 
In order to collect data regarding the most important factors associated with adopting a CMS, the research developed 
a 41-item survey. The questions focus on the primary factors reported in the literature: Ease of use, perceived 
usefulness, self-efficacy, training and support, instructor control, time and work-load issues, organizational culture, 
reward structure, prestige, instructor discipline, pedagogical approach, triability, observability, visibility, relative 
advantage, voluntariness, and CMS flexibility, reliability, and consistency. A four-point Likert scale was used to 
collect faculty members’ responses (1 = “strongly disagree,” 4 = “strongly agree”). 

Data Collection 

An invitation was sent to all faculty members teaching during the fall 2013 semester at both of the 
community colleges. The email invitation contained a brief overview of the nature of the research and a request for 
participation. A response deadline of 10 business days was also included in the invitation email. Invitations were 
sent to approximately 198 faculty members teaching at the small rural community college, approximately 245 
faculty members teaching at the medium rural community college and 1000 teaching at the very large urban 
community college. A follow-up invitation was sent out after 6 days to those who had not responded to the initial 
invitation. 
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Data Analysis 


A Rasch analysis using the computer program WINSTEPSTM was conducted to examine uni- 
dimensionality, item and person fit, difficulty, separation, and reliability and validity, and multi-dimensionality. 
Additionally, an analysis using FacetsTM was conducted to attempt to detect any difference between faculty in 
different community college settings and between part-time and full-time faculty members regarding CMS adoption 
factors. 


Results 

Three community colleges in the Midwest were chosen based on their Carnegie location setting classification. One 
community college was based in a rural setting, another was located in a suburban setting, and the third was based in 
an urban setting. The researcher contacted administrators in the instructional area at each of the community colleges 
to seek approval to conduct research and seek assistance with emailing the part-time and full-time faculty members 
who were teaching at each respective college during the fall 2013 semester. After the researcher received approval 
from the three community colleges, an email was sent to the administrative contacts at each college that contained 
instructions for sending out the email invitation. This email included information about the nature of the research, 
the duration of time that the questionnaire would be available. Five days after the initial questionnaire invitation 
email had been sent, a reminder email was sent to faculty members at the three community colleges encouraging 
their participation. Due to low initial response rates, a third reminder email was sent to faculty at all three 
community colleges two days prior to the scheduled end date for the questionnaire. After the questionnaire response 
collection period ended, the researcher calculated the response rates from the three community colleges. One 
hundred and seventy emails were sent to faculty members at the rural community college, and 61 responded for a 
response rate of 35.8%. Two hundred and thirty-seven emails were sent to faculty members at the suburban 
community college, and 91 responded for a response rate of 38.3%. Finally, 741 emails were sent to faculty 
members at the urban community college, and 190 responded for a response rate of 25.6%. Overall, the researcher 
solicited 1,148 community college faculty members at three different community colleges, and a total of 338 
responded to the questionnaire for a response rate of 29.4%. 

The Rasch analysis of the presented institutional dimension construct map revealed that faculty members 
found it easiest to respond positively to those items related to the advantages of using a CMS, awareness of the 
benefits of using a CMS, how a CMS increases teaching effectiveness, and how CMS use allows for greater control 
over instructional materials. These results suggest that factors that most influence faculty adoption of a CMS are 
related to perceptions of advantages associated with CMS use, such as the ability of a CMS to enhance teaching 
effectiveness and increase control over instructional content. The results support previous research by Chang (2008) 
suggesting that faculty members’ ability to align their teaching approaches and philosophies with new technology, 
such as a CMS, influenced adoption and use. Rasch analysis of the presented personal characteristic dimension 
construct map revealed that faculty members responded positively related to technology self-efficacy, the 
advantages of using a CMS, awareness of the benefits of using a CMS, and the awareness of professional 
development opportunities. Rasch analysis of the presented institutional dimension construct map revealed that 
faculty members found the most difficult to respond to positively related to the college rewarding faculty members 
for using a CMS. Data were analyzed using a multi-faceted Rasch approach to account for additional factors, 
including work status. Both dimensions were analyzed through the multifaceted model to assess possible differences. 
Rasch analysis of the presented institutional dimension revealed a 0.04 measure statistic for full-time faculty and a - 
0.04 for part-time faculty. The standard error statistic for full-time faculty members was 0.11 and 0.09 for part-time 
faculty members. This indicated that there were no statistically significant differences at the 95% confidence level 
regarding CMS adoption factor differences among part-time faculty members and full-time faculty members in the 
institutional-themed group. 

Rasch analysis of the presented personal characteristics dimension revealed a 0. 1 8 measure static for full- 
time faculty members and a -0.18 for part-time faculty members The standard error statistic for full-time faculty 
members was 0.10 and 0.07 for part-time faculty members. This indicated that there was a statistically significant 
difference at the 95% confidence level regarding CMS adoption factor differences among part-time faculty members 
and full-time faculty members in the personal-themed group. Also, it appears that more suburban part-time faculty 
feel that a CMS is easier to learn to use compared to their full-time colleagues. Further, it appears that more part- 
time suburban faculty feel that a CMS fits with their teaching style. To a lesser degree, more part-time suburban 
faculty members perceive that a course management system is compatible with their teaching philosophy. Again, 
urban faculty cross-tabulated responses showed little difference between part-time and full-time faculty. 


161 



Data were similarly analyzed using a multi-faceted Rasch approach to account for additional factors, 
including the geographic location of the colleges. Both dimensions were analyzed through the multifaceted model to 
assess possible differences. Rasch analysis of the presented institutional dimension revealed a 0.59 measure statistic 
for rural, a -0.24 measure statistic for suburban, and a -0.35 measure statistic for urban. The standard error statistic 
was 0.16 for rural faculty members, 0.13 for suburban faculty members, and 0.09 for urban faculty members. This 
indicated that there were statistically significant differences at the 95% confidence level regarding CMS adoption 
factor differences between rural, suburban, and urban community college faculty members. 

Based on these results, it is possible to identify which factors are related to the specific community college 
setting: rural, suburban, or urban. Since the rural measure statistic was 0.59 and the suburban measure statistic was - 
.24, it is likely that rural and suburban faculty members favored these question items. Rasch analysis of the 
presented personal characteristics dimension revealed a 0.26 measure statistic for rural, a -0.04 measure statistic for 
urban, and a -0.21 measure statistic for suburban. The standard error statistic was 0.13 for rural faculty members, 
0.08 for urban faculty members, and 0.11 for suburban faculty members. This indicated that there were statistically 
significant differences at the 95% confidence level regarding CMS adoption factor differences between rural, 
suburban, and urban community college faculty members. 

It is clear from these findings that specific CMS adoption factors resonated more so with community 
college faculty than did other adoption factors. The notion of rewarding faculty for adopting a course management 
system into their instructional practices resonated the most with community college faculty in this study. 
Additionally, the research findings also supported how faculty perceptions of relative advantage, CMS benefits, 
professional development and support, as well as control of instructional materials are positive factors associated 
with CMS adoption. The findings here also revealed that there are difference between full-time and part-time 
community college faculty members and their perceptions of course management system adoption. Specifically, the 
research findings here showed that part-time faculty at rural and suburban community colleges reported that it is 
easier to use a CMS, as well as to learn to use a CMS. Finally, the research findings also revealed differences among 
community college faculty members based on location setting. Specifically, along with the differences comparing 
part-time and full-time faculty, the findings revealed that rural and suburban community college faculty members 
responded more favorably to the question items related to recognition of the advantages of using a CMS, recognition 
of the benefits of using a CMS, and how a CMS enhances teaching effectiveness. Conversely, more urban and 
suburban faculty member found it more difficult to respond favorably to question items related to rewards for using 
a CMS. 


Conclusions 

The following conclusions were drawn from the data analysis: 

Research Question 1: What factors most influence faculty members at two-year community colleges to adopt 
a course management system? 

Faculty members in the institutional dimension group indicated that they found it easiest to respond favorably to 
those items related to (a) the advantages of using a CMS, (b) awareness of the benefits of using a CMS, and (c) how 
a CMS increases teaching effectiveness and allows for greater control over instructional materials. These results 
suggest that the factors that most influence faculty adoption of a CMS are related to perceptions of advantages 
associated with CMS use, as well as how a CMS enhances teaching effectiveness and control over instructional 
content. 

Faculty members in the personal characteristic dimension group indicated that they found it easiest to respond 
favorably to those items related to (a) technology self-efficacy, (b) the advantages of using a CMS, (c) awareness of 
the benefits of using a CMS, and (d) the awareness of professional development opportunities. These results suggest 
that the factors that most influenced faculty adoption of a CMS are related to self-efficacy, awareness of advantages 
and benefits of using a CMS, and professional development. 

Faculty in both groups agreed on the advantages of using a CMS and awareness of the benefits of using a CMS, 
which supported Rogers’ (2003) theory that the perception of relative advantage favorably influenced adoption. 

Research Question 2: What factors most inhibit faculty members at two-year community colleges from 
adopting a course management system? 

Faculty members from the institutional dimension group indicated that the most difficult item to respond to 
favorably related to the college rewarding faculty for using a CMS. These results suggest that a reward structure or 
lack of reward structure inhibits faculty from adopting a CMS. Additionally, these results support previous research 
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that noted how rewards influence technology adoption (Al-Shboul, 2011; Gautreau, 2011; Zayim, Yildiriim, & Saka, 
2006). 

Faculty in the personal characteristics dimension group indicated that the item faculty members found the most 
difficult to respond to favorably also related to the college rewarding faculty for using a CMS. These results were 
similar to what faculty in the institutional-themed group reported. 

Both groups of faculty members were in agreement regarding the need for a reward structure to influence CMS 
adoption, which supported previous research that noted how rewards influence technology adoption (Al-Shboul, 
2011; Gautreau, 2011; Zayim, Yildiriim, & Saka, 2006). 

Research Question 3: Are the course management system adoption factors different for part-time and full- 
time faculty members at two-year community colleges? 

For faculty in the institutional-themed group, there were no statistically significant differences found in this 
study. For faculty in the personal-themed group, there were statistically significant differences found between part- 
time and full-time faculty participants. Based on the study findings it appears that rural part-time faculty participants 
feel that a course management system is easier to use compared to their full-time colleagues. Also, it appears that 
more rural part-time faculty feel that a CMS is easier to learn to use compared to their full-time colleagues. To a 
lesser degree, it also appears that more part-time rural faculty feel that a CMS fits with their teaching style and is 
compatible with their teaching philosophy. 

In the suburban community college setting, similar to the rural community college setting, it is appears that 
suburban part-time faculty feel that a course management system is easier to use compared to their full-time 
colleagues. Also, it appears that more suburban part-time faculty feel that a CMS is easier to learn to use compared 
to their full-time colleagues. Further, it appears that more part-time suburban faculty feel that a CMS fits with their 
teaching style. To a lesser degree, more part-time suburban faculty members perceive that a course management 
system is compatible with their teaching philosophy. Again, urban faculty cross-tabulated responses showed little 
difference between part-time and full-time faculty. 

Research Question 4: Are the course management system adoption factors different for rural and suburban 
community college faculty? 

Based on the responses of the institutional-themed group, it is likely that rural faculty members favored the 
items related to (a) recognition of the advantages of using a CMS, (b) recognition of the benefits of using a CMS, 
and (c) recognition of how a CMS enhances teaching effectiveness. These results support Rogers’ (2003) theory that 
the perception of relative advantage favorably influenced adoption. Additionally, these findings support previous 
research that noted how perceptions of technology usefulness positively influence technology adoption (Anderson, 
2003; Halawi & McCarthy, 2007; Holden & Rada, 2011; Kultur, 2009; Rogers, 2003). Further, urban faculty 
members from this group responded unfavorably to the item related to the college rewarding faculty members who 
use a CMS. This is consistent with prior research noting that rewards influence technology adoption (Al-Shboul, 
2011; Gautreau, 2011; Zayim, Yildiriim, & Saka, 2006). 

Results from the analysis of the responses of faculty members from the personal-themed group indicated 
that rural faculty members responded favorably to items related to (a) technology self-efficacy, (b) recognition of the 
advantages of using a CMS, and (c) the availability of professional development opportunities to learn more about 
CMSs. These results support prior research noting how self-efficacy (Al-Shboul, 2011; Gautreau, 2011; Halawi & 
McCarthy, 2007; Holden & Rada, 2011; Jarrahi, 2010; Kultur, 2009), recognition of advantages (Roger, 2003), and 
the availability of professional development and support (Al-Shboul, 2011; Gautreau, 2011; Kultur, 2009) influence 
technology adoption. 

Additionally, suburban faculty members in personal-themed group responded unfavorably to the items 
related to (a) the college rewarding/compensating faculty members who use a CMS, (b) increasing CMS use if the 
college offered more support, and (c) the awareness of how others are using a CMS. This supported the literature 
noting that rewards influence technology adoption (Al-Shboul, 2011; Gautreau, 2011; Zayim, Yildiriim, & Saka, 
2006) and the fact that awareness of how others are using technology influenced adoption (Rogers, 2003). 
Conversely, for urban faculty members in the personal-themed group the notion of support was not a factor in 
increasing CMS usage, which contradicts previous literature on this topic (Al-Shboul, 2011; Gautreau, 2011; Kultur, 
2009). 
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Discussion 


According to the overall results of this study for the institutional-themed group, faculty members found it 
easiest to respond favorably to those items related to (a) the advantages of using a CMS, (b) awareness of the 
benefits of using a CMS, and (c) how a CMS increases teaching effectiveness and allows for greater control over 
instructional materials. Faculty members from the institutional-themed group also indicated that the most difficult 
item to respond to favorably related to the college rewarding faculty for using a CMS. 

Further, when examining difference between faculty members in community college settings, rural faculty 
members in in the institutional-themed group favored the items related to (a) recognition of the advantages of using 
a CMS, (b) recognition of the benefits of using a CMS, and (c) recognition of how a CMS enhances teaching 
effectiveness, while urban faculty members in the institutional-themed group responded unfavorably to the item 
related to the college rewarding faculty who use a CMS. 

The overall results of this study for the personal-themed group indicated that the items faculty members in 
this group found the easiest to respond to favorably related to (a) technology self-efficacy, (b) the advantages of 
using a CMS, (c) awareness of the benefits of using a CMS, and (d) the awareness of professional development 
opportunities. Conversely, faculty members in personal-themed group indicated that the item most difficult to 
respond to favorably related to the college rewarding faculty members for using a CMS. 

Additionally, when examining difference between faculty members in community college settings, rural 
faculty members from the personal-themed group responded favorably to items related to (a) technology self- 
efficacy, (b) recognition of the advantages of using a CMS, (c) the availability of professional development 
opportunities to learn more about CMSs. Flowever, suburban faculty members from the personal-themed group 
responded unfavorably to the items related to (a) the college rewarding/compensating faculty who use a CMS, (b) 
increasing CMS use if the college offered more support, and (c) the awareness of how others are using a CMS. 

Both groups of faculty members, those from the institutional-themed group and those from the personal-themed 
group, were in agreement regarding the advantages of using a CMS, awareness of the benefits of using a CMS, and 
the need for a reward structure. Both groups of faculty members also reported that rural faculty members recognized 
the advantages of using a CMS. Urban faculty members in institutional-themed group and suburban faculty in 
personal-themed group reported similar results related to a lack of rewards from the college for using a CMS. 

Differences between part-time and full-time faculty members. The results of this study refute previous 
researcher that raised concerns related to the high proportion of part-time instructors at community colleges, 
including compensation issues and how part-time faculty are poorly integrated and do not have the same access to 
technology as do full-time faculty members (Jackowski & Akroyd, 2010). Additional research noted how the high 
proportion of part-time instructors is more common at urban and suburban community colleges than at rural 
community colleges, which tend to hire more full-time faculty (Eddy, 2007). The results of this study revealed 
statistically significant differences between full-time and part-time faculty members in the personal-themed group. 

Based on the results of the multi-faceted Rasch analysis, coupled with an examination of the cross- 
tabulated survey results, it is appears that rural part-time faculty feel that a course management system is easier to 
use compared to their full-time colleagues. Also, it appears that more rural part-time faculty feel that a CMS is easier 
to learn to use compared to their full-time colleagues. To a lesser degree, it also appears that more part-time rural 
faculty feel that a CMS fits with their teaching style and is compatible with their teaching philosophy. 

Further, in the suburban community college setting, similar to the rural community college setting, it is 
appears that suburban part-time faculty feel that a course management system is easier to use compared to their full- 
time colleagues. Also, it appears that more suburban part-time faculty feel that a CMS is easier to learn to use 
compared to their full-time colleagues. Further, it appears that more part-time suburban faculty feel that a CMS fits 
with their teaching style. To a lesser degree, more part-time suburban faculty members perceive that a course 
management system is compatible with their teaching philosophy. Again, urban faculty cross-tabulated responses 
showed little difference between part-time and full-time faculty. 

It is possible that because previous researchers had identified concerns about integrating part-time faculty 
members and increasing their access to technology, colleges have developed initiatives to better acclimate part-time 
faculty members to campus life and assist them with technology access. Additionally, it is possible age affected 
perceptions and use of course management systems by the participants in this study as cross-tabulated results 
showed that more full-time faculty at the rural and suburban college reported that they had 10 or more years of 
college-level teaching experience, 72% and 62.5% respectively, compared to their part-time colleagues, 56.8% and 
45.2% respectively. This supports previous research by Meyer (2009) that noted how age affects technology use, 
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with older faculty perhaps finding it more difficult learn and use new technology. Finally, it also may be that the 
ever-increasing prevalence of smartphone and tablet devices is ameliorating the previously reported access issues. 

Community college setting differences. Research has noted differences between rural, urban, and 
suburban community colleges (Eddy, 2007; Pennington, Williams & Karvonen, 2006; Sink, Jackson, Boham & 
Shockley, 2004). Much of their research has focused on the difficulties rural community colleges face regarding 
funding, geographical restrictions, and technology support. Pennington, Williams, and Karvonen (2006) cited 
budgetary challenges faced by rural community colleges as reasons for these difficulties. These researchers further 
noted that geographic barriers exist at rural community colleges, which compounds the problem of providing 
funding and providing adequate technology and support. Additionally, rural faculty support professionals reported 
less interest in advancing new initiatives in teaching than did respondents from urban community colleges (Eddy, 
2007). Finally, researchers noted that affordable Internet access is available for urban and suburban colleges but not 
for many rural colleges (Sink, Jackson, Boham, & Shockley, 2004). This study found statistically significant 
differences between faculty members in different community college settings. 

Rural Community College Setting. Results from this study found that rural faculty members from the 
institutional-themed group responded favorably to the items related to (a) recognition of the advantages of using a 
CMS, (b) recognition of the benefits of using a CMS, and (c) the ability of a CMS to enhance teaching effectiveness. 
However, urban faculty members from this group responded unfavorably to the item related to the college rewarding 
faculty members who use a CMS. Rural faculty members from the personal-themed group responded favorably to 
items related to (a) technology self-efficacy, (b) recognition of the advantages of using a CMS, and (c) the 
availability of professional development opportunities to learn more about CMSs. 

Additionally, it is appears that rural part-time faculty feel that a course management system is easier to use 
compared to their full-time colleagues. Also, it appears that more rural part-time faculty feel that a CMS is easier to 
learn to use compared to their full-time colleagues. To a lesser degree, it also appears that more part-time rural 
faculty feel that a CMS fits with their teaching style and is compatible with their teaching philosophy. 

The findings here seem to indicate that the rural faculty members in this study were more technologically 
advanced than those in previous research. According to Eddy (2007) rural faculty support professionals reported less 
interest in advancing new initiatives in teaching than did respondents from urban community. Additionally, it does 
appear from this study that part-time rural faculty members reported that using a course management system was 
easier compared to full-time rural faculty members. Again, this may be related to the age of full-time faculty 
compared to part-time faculty, as more full-time faculty reported having 10 or more years of teaching experience 
compared to part-time faculty members. Furthermore, the differences could be related to the differences in teaching 
departments as more full-time rural faculty reported being from either the health sciences, social sciences or 
vocational/trades departments compared to part-time faculty that reported being from business, health sciences or 
social sciences departments. 

Suburban Community College Setting. Overall, suburban faculty in this study favored question items 
related to recognition of the advantages of using a CMS, recognition of the benefits of using a CMS, and how a 
CMS enhances teaching effectiveness. Additionally, suburban community college faculty members appear to feel 
that a course management system is easier to use compared to their full-time colleagues. Also, it appears that more 
suburban part-time faculty feel that a CMS is easier to learn to use compared to their full-time colleagues. Further, it 
appears that more part-time suburban faculty feel that a CMS fits with their teaching style. To a lesser degree, more 
part-time suburban faculty members perceive that a course management system is compatible with their teaching 
philosophy compared to their full-time counterparts. 

The findings here indicate that the suburban faculty members in this study had similar perceptions 
regarding CMS adoption compared to rural faculty. Additionally, similar to rural faculty in this study, part-time 
suburban faculty members reported that using a course management system was easier compared to their full-time 
counterparts. Again, this may be related to the age of full-time faculty compared to part-time faculty, as more 
suburban full-time faculty reported having 10 or more years of teaching experience compared to part-time faculty 
members. 

Urban Community College Setting. Urban faculty members from the institutional-themed group indicated 
that the most difficult question item for them to respond to favorably was related to the college rewarding faculty 
members who use a course management system. Overall, urban faculty members in this study were female, part- 
time and were from a humanities, science, business or health science department. Perhaps the majority part-time 
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status of urban faculty in this study was the more influential factor related to the group indicating the need for rewards 
in CMS adoption. This would confirm previous research that noted how part-time faculty are less integrated into the 
college and poorly compensated (Jackowski & Akroyd, 2010). Further, the findings here support previous research 
that noted how urban institutions tend to employ more part-time faculty compared to rural community colleges (Eddy, 
2007). 


Community College Administrators. The results of this study supported previous research related to 
rewards influencing CMS adoption (Al-Shboul, 2011; Gautreau, 2011; Zayim, Yildiriim, & Saka, 2006). Interestingly, 
this study found that faculty participants from urban and suburban community college settings perceived rewards to 
be a greater factor in CMS adoption than did rural community college faculty participants. In whatever community 
college setting, it seems clear that faculty members perceive rewards as a factor that influenced CMS adoption. 
Community college administrators who wish to increase faculty use of CMSs should consider a reward structure as a 
way to increase CMS usage, as well as other factors. This study encourages community college administrators to: 

1 . Consider introducing rewards such as merit pay, release time, recognition programs, or adjustable teaching 
workloads to encourage CMS adoption. 

2. Develop professional development programs that showcase how current CMS faculty adopters are using 
CMSs in their instructional practices. 

3. Promote CMS use to faculty by highlighting the advantages of efficiency, grade transparency, improved 
communication, greater control and flexibility with distributing course materials and reduced printing costs, 
all benefits associated with course management system use. 

4. Ensure adequate support services are available for new CMS users, as well as advanced CMS users. 

5. Choose course management systems based on their ease of use and usefulness. 

State leaders in higher education. According to Louis Soares (2013), vice president for Policy Research and 
Strategy at the American Council on Education, “...the current state policy environment does not encourage the 
adoption of these technologies (learning technologies) in a generative way in which proven practice informs policy 
information” (p. 72). The findings of this study are supportive of learning technology, specifically course 
managements systems, and encourage the increased use of these types of systems. In order for community college 
leaders to be able to initiate policies to increase CMS adoption at their campuses they need support from their 
respective states. 

Summary 

The faculty members in this study reported that being rewarded is a factor that may influence them to 
increase their CMS adoption. This is especially true of the faculty members from urban and suburban community 
colleges. The results of this study supported Rogers’ (2003) theory regarding relative advantage and its positive 
influence on technology adoption among faculty members. This was especially true of faculty members from rural 
community colleges. Faculty members’ perceptions of (a) technology self-efficacy, (b) control, and (c) training and 
development also were factors in CMS adoption, as were other factors to a lesser degree, based on the community 
college setting. The results of this study indicated that not only are there specific factors that influence CMS faculty 
adoption but also that the community college setting also influenced the prominence of some factors over others. 
Community college administrators who wish to increase their CMS adoption should examine their current reward 
structure for encouraging CMS adoption. Community college administrators should consider some type of 
recognition, pay, stipend or adjusted workload for faculty who adopt a CMS into their teaching practices. When 
community college administrators are promoting CMS adoption, they also should emphasize the factors of relative 
advantage and control and ensure that their college has adequate faculty CMS training and support. 

This study explored a variety of technology adoption factors and their importance in CMS adoption among faculty 
members at three different community colleges situated in three different Carnegie Classification settings. The 
results of this study provided insight into the influence of different adoption factors on faculty members from 
different community colleges in different settings. Future research should account for the absences of non-users and 
for the division of questionnaire items that were either personal-themed or institutional-themed to provide a more 
focused analysis on this topic. 

If the trend of technology in education continues, this study and other similar studies will provide important 
contributions that will benefit college and university faculty members, students, and administrators. Until colleges 
and universities achieve complete participation from faculty members regarding CMS use, research on this topic 
should continue so that the perceptions and needs of faculty members can be captured and then addressed through 
college and university policies and initiatives. 
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Peer-Led Online Discussion in Compressed Courses: 

Do the Benefits Outweigh the Logistical Risks? 

Penny Thompson 

Oklahoma State University 

Abstract 

This study explored whether there were differences in the quantity and quality of online discussion posts when 
identical prompts were provided by peers or by the instructor. Discussions from two years of an 8-week course, 
representing the peer-led and instructor-led conditions, were compared. The peer-led condition produced deeper 
levels of reply posts and a higher percentage of helpful critique points compared to the instructor-led condition, but 
there were no statistically significant differences between the two conditions. Results suggest the need for continued 
research into the value of peer-led discussion in compressed courses where instructor-led discussion might be more 
efficient. 

Teaching presence, defined by Anderson, Roarke, Garrison, and Archer (2001) as “the design, facilitation, 
and direction of cognitive and social processes for the purpose of realizing. . .learning outcomes” (p. 5), is an 
important element in supporting student learning in online environments. One important component of teaching 
presence in most online courses is the facilitation of asynchronous online discussions. However, as too many 
instructor posts can suppress student participation (Mazzolini & Maddison, 2007), instructors face a difficult balance 
when deciding how much to participate in online discussions. One solution to this challenge is to have peer 
facilitators take on part of the teaching role (Anderson et al., 2001), but assigning this task to students may pose 
logistical difficulties, particularly in courses that run for less than a full semester. The time and effort student 
facilitators spend preparing a discussion prompt and monitoring a discussion forum may come at the expense of time 
spent processing other course content and reflecting on their own learning. Instructors of short-duration courses 
therefore need assurance that the benefits of placing this extra responsibility on the students outweigh the risks. 

This study explored whether the quantity and quality of online discussion posts differed when peers, rather 
than the instructor, provided the discussion prompt in an 8-week online course. Discussions from two consecutive 
years of the course, one using an instructor-led discussion and the other using a peer-led discussion, were analyzed 
at two levels. First, the two conditions were compared for number and length of posts and depth of discussion 
threads. Then the quality of peer feedback between the two conditions was compared using Costa and Kallick's 
(1993) Critical Friends guidelines. 

Peer-led discussion 

Research has shown that students are more engaged in online discussions when they are acting in the role 
of peer facilitator (Poole, 2000; Zha & Ottendorfer, 2011), but few studies have explored whether the presence of 
peer facilitators affects learning or engagement among other students during the time they are not acting as 
facilitators. Seo (2007) found that peer moderation led to more student interaction and more substantive posts 
compared to unmoderated discussion. Furthermore, Hew (2015) found that although many students preferred 
instructor-led discussions, those who preferred peer-led discussions reported feeling more comfortable sharing their 
own views and were less worried about saying something “wrong” when their peers led the discussion. The author 
noted, however, that Asian cultural norms might have influenced this result, so it is unknown whether students from 
other regions or cultures would have similar reactions to peer facilitation. 

The current study addressed the question of how the use of peer discussion leaders influenced the online 
posting behavior of the students not acting as leaders. While tallies of posts, word counts, and thread levels are not 
important in and of themselves, they indicate the level of interaction between students, and this interaction provides 
a necessary (though not sufficient) condition for constructive peer feedback to occur. 

Feedback from Critical Friends 

Costa and Kallick (1993) describe the Critical Friend as a person who “takes the time to fully understand 
the context of the work presented and the outcomes that the person. . .is working toward” and then acts as “an 
advocate for the success of that work” (p. 50). A successful exchange of peer feedback with Critical Friends requires 
thoughtful participation from all parties. The learner’s role is to fully describe the project and request feedback, 
clearly communicating what he or she hopes to accomplish. Critical Friends provide different types of feedback at 


169 



different stages of the critique process. First, they ask clarifying questions to make sure they fully understand the 
process or project the learner is presenting. Next, they provide detailed feedback that highlights the unique 
significance or value of the work. Finally, they pose challenging questions and offer suggestions for how the learner 
might improve the work. Since the instructor has a strong influence over the climate of a classroom (Rocca, 2010), 
instructor involvement in discussion posts could influence whether students engage with each other and act as 
Critical Friends or hold back expecting the instructor to provide all needed feedback. The current study explored 
whether the number and percentages of post conforming to the Critical Friends guidelines differed between the peer- 
led and instructor-led conditions. 


Research Questions 

This study addressed the following research questions: 

1 . Is the source of a discussion prompt (peers versus instructors) associated with differences in the 
number of posts, length of posts, or thread depth in an online discussion? 

2. Is the source of a discussion prompt (peers versus instructors) associated with differences in the 
quality of peer feedback as defined by the Critical Friends (Costa & Kallick, 1993) guidelines? 

Methods 

Participants in this study were students (mostly working professionals in the P-12 schools or higher 
education) in an accelerated 8-week online master’s level course centered on the design of online courses. In Year 1 
of the study there were 27 students enrolled (9 male, 18 female) and in Year 2 there were 18 students (7 male, 9 
female). Because this research project was conceived after both years of the course had already concluded, 
permission was sought and obtained from the university’s Institutional Review Board to analyze and report only 
aggregate statistics on the students’ discussion posts, and not to contact the students to collect additional data. 
Therefore, detailed demographic information about the participants is not available. 

Discussion posts from a single week of the 8-week course were analyzed. The assigned task was to produce 
a detailed plan for an educational online activity the students could use in their own teaching, and the goal of the 
discussion was to get feedback from their peers to help develop and improve their plan. Students responded in an 
asynchronous discussion forum to the following prompt: 

Your textbook discussed several activities suitable for online course. Choose one that you 
have not used before and describe how you might use it to teach online in your subject 
area. 

In Year 1 this prompt was written by a group of six students in the class, who were required to develop a 
discussion prompt and submit it to the instructor by Tuesday of their assigned week. The instructor then posted the 
prompt to the course website and the other students in the class were required to respond and participate in 
discussion by the following Sunday at midnight. The students who posed the prompt then acted as discussion 
moderators. There were no specific roles assigned to the student moderators, other than to “provide some sort of 
follow-up, such as responding to some of the posts and/or providing a summary at the end.” The six student 
moderators were not required to respond to the prompts as participants in the discussion. Thus, there were 21 
students responding to the prompt over a period of six days (Tuesday morning to Sunday evening). 

In Year 2 the instructor decided to reduce the amount of group work in the course for logistical reasons, 
and therefore provided all of the discussion prompts instead of assigning this responsibility to groups of student 
leaders. For this particular week, however, she chose to reuse the prompt provided by the Year 1 students. She 
posted the prompt on Sunday before the beginning of the fourth week of the term, giving the students eight days to 
respond. The instructor moderated the discussion, and students did not know that the prompt had been suggested by 
a group of their peers from the previous year. A total of 1 7 students participated in the discussion. 

This change to the course provided a quasi-experimental condition between the two years. The course, 
instructor, and discussion prompt were identical, so the only differences were the source of the prompt (peers versus 
the instructor) and the fact that students in the instructor-led condition received the prompt earlier in the week and 
thus had two additional days to respond. 

Although the students were not explicitly instructed to use the Critical Friends guidelines when responding 
to peers (as this study used archival data not originally collected for research purposes), the assignment instructions 
referenced the Costa & Kallick (1993) article and directed students to “avoid just saying ‘good job’ or ‘I agree’ and 
instead give [your classmates] some specific feedback or pose a question that will help them to continue to develop 
their ideas.” 
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Data Analysis 


All discussion threads from the fourth week of the term were extracted from the course websites, de- 
identified, and imported into the QDAMiner software for coding. The unit of analysis was the entire student post, 
although more than one code was occasionally assigned to posts longer than a paragraph. Posts were coded first 
based on their place in the threaded discussion, using the code P for an initial post, R1 for a student reply to a P, and 
R2-R5 for student replies to the preceding R. Once this level of coding was complete all R1-R5 posts were also 
coded as (1) clarifying questions, (2) detailed feedback about the value of the work, or (3) critique points to help 
improve the project, based on the guidelines of Costa and Kallick (1993). Posts that did not fit into one of these three 
categories were coded as “other.” 

After the coding was complete, data were analyzed quantitatively by counting the number of posts of each 
type and the number of words in each post. The numbers of posts at each reply level (codes R1 - R5) were compared 
between years using the nonparametric Mann-Whitney U Test, due to the small sample sizes and non-normal 
distributions. The average number of replies per student was also calculated. Word counts were compared between 
years using Independent samples t-tests for all levels of post for which there was sufficient sample size. The 
percentage of posts conforming to the Critical Friends guidelines was also calculated for each year, and the number 
of posts between years was compared using the Mann-Whitney U Test. 

Findings 

In both years students entered only one primary post (code P), but the number of reply posts (codes R1-R5) 
varied between years and reply levels, as shown in Table 1. At the R1 level the students in the instructor-led 
condition had higher average replies per student, but for all lower reply levels (R2-R5) and total replies (R1-R5) the 
average replies per student were higher in the peer-led condition. 

Table 1: Posting Levels 

Number of Posts Number of Posts 
Per Student 


Code 

Year 1 

Year 2 

Year 1 

Year 2 

P 

21 

17 

1.00 

1.00 

Rl 

24 

21 

1.14 

1.24 

R2 

18 

11 

0.86 

0.65 

R3 

9 

2 

.43 

0.18 

R4 

1 

0 

.05 

0 

R5 

1 

0 

.05 

0 

R1-R5 

53 

34 

2.52 

2.00 


A Mann-Whitney U Test of the combined reply posts, however, showed that the number of replies in Year 1 (Mdn = 
1.0) was not significantly different from the number of replies in Year 2 (Mdn = 2.0), U = 173.5, p = .885. 

Word counts for each level of post were also computed and compared between the two years. Word counts 
met the conditions for parametric statistical analysis and were therefore compared using independent samples t-tests. 
There was no significant difference in the length of “P” posts between Year 1 (M=458, SD=323) and Year 2 
(M=3 14, SD=129); t(36)=1.87, p = .072, and no significant difference in the length of “Rl” posts between Year 1 
(M=89, SD=102) and Year 2 (M=98, SD=136); t(36)=-.24, p = .81. For codes R2 - R5 there were too few posts to 
analyze. 

The analysis based on the Critical Friends guidelines, shown in Table 2, showed that the majority of replies 
from both groups were consistent with the guidelines (i.e., coded as clarifying questions, comments on the value of 
the work, or critique points) with a smaller percentage falling into the "other" category (e.g., simple compliments, 
personal anecdotes, or comments purely social in nature). 
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Table 2: Critical Friends Posts 

Y ear 1 Y ear 2 



Number 

Percent 

Number 

Percent 

Question 

14 

22 

4 

14 

Value 

29 

45 

15 

52 

Critique 

18 

28 

4 

14 

Other 

3 

5 

6 

20 

Total 

64 

100 

29 

100 


Note that in the peer-led group (Year 1), 95% of the posts were coded as critical friend posts and only 5% 
were coded as "other." In the instructor-led (Year 2) condition only 80% of the posts were coded as Critical Friends 
posts while 20% were coded as "other." A Mann- Whitney U Tests, however, showed that the number of Critical 
Friends posts for Year 1 (Mdn = 2.0) was not significantly different from Year 2 (Mdn = 1.0), U = 137, p = .232. 

Discussion 

This study explored whether students in an online course responding to discussion prompts posed by 
classmates posted more frequently, entered longer posts, or engaged in deeper levels of replies compared to students 
responding to the same prompt posed by the instructor, and whether they offered more productive feedback as 
defined by the Critical Friends (Costa & Kallick, 1993) guidelines. While post counts, word counts, and thread depth 
alone are not clear measures of student engagement, they do suggest a greater level of interaction and thus more 
opportunity for constructive feedback. In the current study only students in the peer-led condition produced any 
discussion posts deeper than the third level, but numbers of posts and word counts were not significantly different 
between the two conditions. The peer-led condition also produced a higher percentage of replies consistent with the 
Critical Friends (Costa & Kallick, 1993) guidelines, though the difference was not statistically significant. 

While previous research has demonstrated the value of student-led discussions (see Mazzolini & Maddison, 
2007; Rocca, 2010; Swan, 2003), the current study explored the value of student-led discussion in an 8-week course. 
In a compressed course it is possible that the time pressure and the need to attend to more course material in a short 
amount of time could outweigh the benefits of increased student leadership. The results of the current study did not 
show significant differences in the number or length of posts or in the quantity of posts consistent with the Critical 
Friends guidelines between the instructor-led and the peer-led conditions. Thus, there is no clear indication that the 
extra responsibility placed on students in the peer-led condition either harmed or helped their learning. Nevertheless, 
the fact that only the peer-led condition achieved thread depth greater than three levels, plus the greater percentage 
of Critical Friends posts in the peer-led condition, suggest that further exploration of this question is warranted. 

This study is limited by its small sample size; a larger sample might have revealed statistically significant 
differences that were not measurable with these small classes. The use of archival course data also limited the ability 
to measure any preexisting differences between the two groups. While the course activities did produce a naturally 
occurring quasi-experiment, with the instructor and course activities held constant and the source (peer versus 
instructor) of the discussion prompt serving as the independent variable, there may be differences in student 
characteristics between the two years that influenced the results. While no firm conclusions can be drawn from the 
findings of this study, the questions raised here need to be explored further in order to provide instructors with 
guidance on how to balance the logistical challenges of compressed courses with the benefits of letting students take 
a leadership role in the community of learners. 
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Abstract 

Registered nurses go through employee orientation training whenever they begin to practice. The 
correlation of nurses and employee orientation training has become such a common occurrence that hospitals and 
other organizations around the country are holding orientation training sessions specifically for the registered 
nurse population (Wells, 1999). Within these orientation programs, there’s a need that has not yet been met. This 
is the need of orientation programs becoming technologically and culturally adept to the needs of nurses. In order 
to meet these needs, it is suggested that orientation programs adopt an online portion within their orientation 
program (Altimier, 2009). With a portion of the orientation program done in an online format, hospital expenses 
decrease, while orientation efficiency and uniformity increase (Altimier, 2009). The benefits of online learning 
outweigh the costs of establishing the program because it has been proven to improve nurse to patient interaction, 
learner satisfaction, employee retention, and the user gains a deeper learning (Altimier, 2009 & Wells, 1999). 

This type of orientation training is vital to the changing nurse organizational culture within the nursing field, 
which indicates that flexibility and convenience is key to training program success. An integrative literature 
review will be conducted in order to review peer-reviewed articles that detail the relationship between nurse 
organizational culture, learning, and technology within nurse orientation training programs. 

Definition of Topic 

Culture, learning, and technology are all different facets that impact the effectiveness of nurse orientation 
training programs. The culture within an organization has a deep impact on the retention of employees within the 
nursing field, as well as in the nurse determining whether the existing culture is able to allow the nurse to continue 
learning, which is vital to a nurse’s success (Cleary, Horsfall, Jackson, Muthulakshmni, & Hunt, 2013; Hunter, 
Spence, McKenna, & Iedema, 2008; Jae & Kim, 2009). Both the culture and learning environment of an 
organization are determined early on within a nurse being employed by a hospital, which is usually within their 
orientation training program. Due to this, organizations have found that incorporating a higher level of technology 
within the orientation training program that adds to employee ease, may be worth considering in order to invest in 
these employees and in return begin to see a reduction in employee turnover (Altimier, 2009; Campbell, 2009). 

Method 

A review of literature was conducted to determine the impact of culture, learning, and technology on nurse 
orientation training programs. A search of articles, both peer-reviewed and full text, was made from multiple 
databases via the University of North Texas’ online library database. Date and database parameters were defined to 
obtain current specific data. Data from the time period of 1999 to 2015 was used and journal articles were retrieved 
from the electronic databases of Health Reference Center Academic Infotrac Database, Sage Journals Online, 
SpringerLink, ProQuest, Science Direct, JSTOR, and Wiley Online Library. To narrow the search, the subject term 
selected was article, the content type selected was journal article, and the language selected was English. 

The first search was compiled using the term “Technology in hospital orientation training.” The second 
search was compiled using the term “Nurses employee orientation training programs.” The third search was 
compiled using the term “Nurses employee culture.” The fourth search was compiled using the term “How do 
nurses learn in the workplace.” 
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Intentional Exclusions 


In order to allow for cohesiveness in the findings of the literature review, studies that involved nursing 
orientation programs that were not geared towards registered nurses were excluded. This will help to get a more 
accurate finding of the impact of culture, learning, and technology on nursing orientation programs specific to 
registered nurses and not licensed practical nurses or licensed vocational nurses. 

Literature Review 

New graduated registered nurses who decide to stay with their first employer a year after graduation ranges 
from 25% to 64% (Friedman, Delaney, Schmidt, Quinn, & Macyk, 2013). Due to this vast difference in retention of 
new registered nurses, hospitals are looking for ways to attract registered nurses, as well as to keep them for more 
than a year (Friedman, et. al, 2013). In order to begin to do this nurse orientation and education programs are 
seeking to increase both efficiency and flexibility (Altimer, 2009; Campbell, 2009; Chestnut & Everhart, 2007; 
Hunter et al, 2008; Wells, 1999). Traditional classroom format orientation received feedback of being long, intense, 
draining, and lacking the consistency of a constant preceptor from new registered nurses completing them 
(Patterson, Bayley, Burnell, & Rhoads, 2010). Nurses are also stating that not only are these type of orientation 
programs containing much repetition about skills that may not apply to their specific department, but not enough 
repetition on skills that are of greater need, such as cardiac care protocol and drug protocol are being given 
(Patterson et. al., 2010). 

Nurses want an orientation program that adapts to their culture and learning needs, which include being 
able to get the information needed in a prompt manner that is also flexible with the needs of the hospital department 
that they will be working in (Campbell, 2009; Wells, 1999). This leads to an increase use of technology within the 
registered nurse orientation program (Altimer, 2009). The decision to include technology within orientation 
programs to adjust to registered nurses’ culture and learning needs has prompted organizations and individuals to 
develop web-based orientation programs (Campbell, 2009). 

Within the state of Virginia, the constructivist learning theory, which stresses the generation of ideas from 
interaction, was used to develop a web-based orientation program for school nurses in order to adjust nurses to the 
setting, overcome the barrier of time, and give the nurses the ability to navigate through the program in a module 
format with the ability of having unlimited access (Campbell, 2009). The orientation addresses the issues of school 
nurse resources within the state of Virginia, the role of the nurse in the school environment, and the standards of 
school nursing practice along with the measures of the standards (Campbell, 2009). 

Another place where web-based orientation programs are being implemented is within specialty areas of 
hospitals. Neonatal departments within hospitals are also developing web-based orientation programs for their 
nurses ( Altimier, 2009). These programs have a goal of providing flexibility in a learning environment, as well as 
providing a platform that leads to a consistency of guidelines, as the web-based format allows for the ability to 
consistently update materials (Altimier, 2009). Altimier (2009) also discusses the benefits of implementing the web- 
based orientation program, which includes “deeper learning, increased interaction, increased sense of learner 
control, economic efficiency, easily updated learning materials, improved learner satisfaction, improved retention, 
and improved learning outcomes.” The course is designed in modules, with an education credit being given at the 
completion of each module, in order to allow the nurse to implement the skills learned in each module and thus 
further increase efficient learning (Altimier, 2009). 

While the number of organizations deciding to develop and implement web-based orientation programs are 
increasing, the impact that this is having on nurses still needs to be measured. The following portion will contain 
the results of the impact of culture, learning, and technology on nursing orientation programs. 

Results 

The impact that instilling a change within orientation programs have on registered nurses has given benefit 
in both nurses’ culture and learning (Altimier, 2009; Campbell, 2009). Programs deciding to make a change within 
their traditional nurse orientation programs have been able to change those 25% to 64% registered nurse retention 
rates up to an increase of 84% to 94% (Friedman et al, 2013). Figure 1 outlines the increase in retention both before 
and after the implementation of a new orientation program. Within the chart there is a marked increase in the 
retention rate of registered nurses after the implementation. There is also a decrease in the gap between the retention 
rates between the lower levels and the upper levels. 
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A New Look at Nurse Orientation Programs 
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Synthesis 

A common theme has arisen from the literature review and the results. A multifaceted approach that 
includes culture, learning, and technology is what the majority of the literature has suggested in order to gain a 
positive impact in registered nurses’ orientation training programs. This includes orientation programs adjusting to 
the needs of the nurses, as their culture needs flexibility and efficiency in order to properly fulfill their duties. This 
has been seen in the ability of orientation programs to garner more technology within the program to increase the 
nurses’ learning. In turn, these improvements of orientation programs leads to better registered nurse employee 
retention as well. 
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Abstract 

This study identified and discussed the factors that challenging instructors’ adopting of digital content in 
higher education. It focused on instructors’ perspectives of adopting e-Textbooks in their classrooms. A quantitative 
research design and descriptive analysis were applied to collect and analysis data from the instructors who were 
teaching in public universities. Findings of this study were the four factors of the challenges that delayed or even 
terminate instructors’ adoption of e-Textbooks in their teaching, which included institutions’ influence on adoption of 
e-Textbooks, instructors’ attitudes towards using e-Textbooks, students’ learning needs with e-Textbooks, and the 
availability of e-Textbooks via publishers. This study also suggested a collaboration of the institutions, instructors, and 
e-Textbooks publishers to promote e-Textbooks in education in the future. 

Introduction 

Among the 50 states of the United States, 32 were in the process of considering adopting digital content in 
education (State Education Policy Center, 2012). With its various fonnats, digital content promotes the 
customization of teaching and learning, improves the flexibility of learning which enables learning to happen at 
any time and any place, and enhances the accessibility to learning materials (Pearson Publishing, 2013). E- 
Textbooks as a kind of digital content refer to both the textbook content and the hardware that run the content, and 
have a variety in terms of definition (Cavanaugh, 2005; Kissinger, 2011; Larson, 2009; Rao, 2003) and format 
(Cavanaugh, 2005; Hawkins, 2000; Kissinger, 2011; Wikipedia, 2013). They can easily be a simple electronic 
form of a textbook, either in PDF, text, or other Web-compatible formats (Lamothe, 2011; Tripathi, & Jeevan, 
2007); they can also be ones with complex digital features, such as word pronunciations, text highlighting, text-to 
speech options, and hypermedia (e.g., video, animations, sound) (Chen, Crooks, & Ford, 2013; Dalton & Palincsar, 
2013; Daniel & Woody, 2013). They provides different solutions for users to personalize their learning content, 
digitalize educational activities, and accommodate diverse learning backgrounds (Wang, 2015). 

Researchers, educators, and e-Textbooks publishers have conducted quite a few pilot studies, projects, and 
researches on e-Textbooks in from preK-12 education to higher education and library in the recent decades. The 
results provided observable evidences of the benefits of using e-Textbooks in education, which included but not 
limited to increasing students’ achievements, creating more interactions, and customizing teaching and learning 
experience (Cavanaugh, 2005; Duffey & Fox, 2012; eTextbook, 2012; Larson, 2010; Lynch, 2013; Nelson & 
Hains, 2010; Rosen & Beck-Hill, 2012; Schachter, 2009; Stone, 2008; Wood, Littleton, & Chera, 2005; Zoellner & 
Cavanaugh, 2013). 

However, the current adoption of e-Textbooks in higher education were not smooth. The type of 
innovation decision, the nature of communication channels, the nature of the social system, and the extent of 
change agents’ promotion efforts in diffusing of the innovation (Rogers, 2003) singly or combined affect the 
adoption result. Some researches indicated there were forces that compelled educators’ adoption of educational 
innovation (Korres, 2011; Rogers, 2003; Weidert, 2012; Zoellner & Cavanaugh, 2013), which made the adopting 
difficult, and brought pressures to instructors who were considering the adoption of educational innovation. The 
upward force from individuals (students) and downward force generated by the institutions set the instructors in 
dilemma and caused their innovation adoption to be full of pressure and intensiveness (Korres, 2011), which made 
instructors faced with “ambiguous tension and uncertainty” (Weidert, 2012, p. 51). 

The purpose of this study was to identify the specific challenges that instructors encountered when 
adopting e-Textbooks in teaching, and enumerate the factors that delay or even terminate the adoption process of e- 
Textbooks in higher education. This study examined the research question, what are the factors that challenging 
instructors’ adopting e-Textbooks in higher education. By surveying the current adoption progress among 
instructors who were teaching in higher education and their perceptions of using e-Textbooks in instructional 
activities, this study presented the main challenges that annoyed instructors when using e-Textbooks, explicated 
how Rogers’ four factors (Rogers, 2003) affect instructors’ adoption of e-Textbooks in teaching, and support 
instructors to make proper decisions of how to adopt e-Textbooks in classrooms. 
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Research Method 


This study employed a quantitative research method and applied descriptive analysis to examine the 
specific factors that challenging instructions adopting e-Textbooks in higher education. 

An open-ended survey questionnaire was designed to collect data from the instructors in the colleges of 
education in 39 public universities in the east south central region (Mississippi, Alabama, Kentucky, and 
Tennessee) in the U.S, and included all the lecturers, assistant professors, associate professors, full professors, 
and others who were teaching face-to-face classes, hybrid classes, or online classes in colleges of education 
where they could access to both e-Textbooks and printed textbooks. Descriptive analysis including percentage 
and frequency calculation was deployed to obtain the frequency of using e-Textbooks in teaching. Open-ended 
responses were recorded by instructors’ self-reported experiences and results were organized by factors based 
upon Rogers’ innovation theory (2003). 


Findings and Results 

Within the 347 out of 3 66 valid responses, 5 1 .4% instructors reported that they “never” used digitalized 
textbooks in their teaching; only 1.9% instructors used e-Textbooks as “always”. Among those who had the 
experience of using e-Textbooks, 7.4% instructors used e-Textbooks “very often”, while 29.8% instructors 
“occasionally” used e-Textbooks for teaching, and 4.4% instructors “fairly” used e-Textbooks. 

Four factors emerged as the challenges that delayed or discontinued instructors’ adopting of e- 
Textbooks in higher education: (1) institutional influence on the adoption of e-Textbooks, among which, 
institutions did not provide devices to distribute e-Textbooks in class, institutions did not offer instructors 
adequate knowledge of using e-Textbooks in teaching, and institution did not allow e-text were the most 
mentioned challenges by the instructors; (2) instructors’ attitudes towards e-Textbooks, among which, 
instructors’ little concerns on e-Textbooks, their preference to printed textbooks, their indifference to the 
formats of textbooks, and they did not use textbooks at all were the most mentioned issues among instructors; 3) 
students’ learning needs with e-Textbooks, among which students’ preference to the printed textbooks and 
students’ inadequate knowledge of e-Textbooks were the most two challenges that complained by instructors; 4) 
the availability of e-Textbooks, among which the unavailability of e-Textbooks via e-Textbooks’ publishers, the 
unsatisfied quality of e-Textbooks, and the limited lifetime of e-Textbooks were the most challenges mentioned 
by the instructors. 

Institutional Influence on the Adoption of E-Textbooks 

An innovation decision could be made by optional, collective, or authority in a social system; usually, a 
decision that made through authority could be acceleration to the entire adoption process (Rogers, 2003). 17.2% 
of the instructors mentioned that “there was limited technological support in [my] department regarding to 
integrating e-Textbooks”. The institutions (public universities), as the authority in this social system, did not 
provide adequate support for instructors to accelerate the adoption process of e-Textbooks; instead, those public 
universities offered fewer or even no resources relating to e-Textbooks, neither technically nor economically. 

No devices to distribute e-Textbooks in class. 28.4 % of all the instructors described their situations of 
not being able to use e-Textbooks due to the lack of the e-Readers. 39% instructors from Alabama mentioned 
that when trying e-Textbooks for teaching, “e -readers are sometime limited during day classes”. Their 
institutions did not prepare an adequate supply of the devices to support the adoption of e-Textbooks in teaching, 
especially when instructors had their intentions to use the technology. Instructors from Tennessee also 
mentioned that, not only the instructors lacked the devices to distribute the content of e-Textbooks but also the 
students did not “have a tablet access” e-Textbooks. If “not all students have access to e-readers” in class, it was 
hard for instructors to distribute teaching content through e-Textbooks. 

Inadequate knowledge of using e-Textbooks. Unfamiliarity with educational innovations declined the 
instructors’ intention of adopting it in their teaching. Instructors realized that “any new technology requires 
institutional support for effective implementation”, so that they were hoping to receive maximum support from 
their institutions. However, the “minimal support from the institution for existing technology” keeps instructors 
away from adopting e-Textbook. “I have limited experience with e-books and it is not anything that our college 
has discussed collectively,” and they ascribed it to “because [users of e-Textbooks] numbers are too small and 
thus not profitable”. Even for those who were using e-Textbooks in class, “sometimes the technology fails in 
class when I need the book”, which disappointed instructors. Institutions did not offer such as workshop or 
training to disseminate the basic knowledge of e-Textbooks or guide instructors of using e-Textbooks in 
teaching, not even provide any effective technical support for implementing e-Textbooks in classes, which 
frustrated instructors’ enthusiasm in adopting e-Textbooks in teaching. 

E-text were not allowed. There were even some institutions that completely rejected e-Textbooks in 
their setting. As reported by instructors from Kentucky, “I would love to use an e-textbook but Educational 
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Leadership has not caught up to this.” It is impossible for instructors to adopt e-Textbooks in teaching if their 
social system blocked the innovation. Those institutions as the social authority actually already made their 
decision on adopting e-Textbooks, that is, rejection. 

Instructors’ Attitude Towards E-Textbooks 

The nature of communication channels also influences the instructors’ adopting of e-Textbooks in their 
teaching, which can be mass media or interpersonal (Rogers, 2003), which means that if e-Textbooks could be 
introduced through colleagues, it would be an easier adoption. However, the findings showed that most 
instructors still rely on printed textbooks in teaching, which made the communication channel of colleagues 
difficult. 

No concerns on e-Textbooks. 14.2% of all the instructors were not interested in e-Textbooks at all. “I 
am not interested in e-Textbooks”, 19% instructors from Mississippi, 17% instructors from Alabama, 12% 
instructors from Kentucky, and 17% instructors from Tennessee offered their attribute towards e-Textbooks. 
Some instructors “never thought about it (e-Textbooks)” or “investigated using them in (their) classes”, while 
some other instructors “haven't considered it before but (were) very open to it”. As the communication channel 
of adopting e-Textbooks in teaching, instructors did not perform as an optimistic role; they, themselves, even 
become a big challenge of using e-Textbooks in teaching. This analyzed result even conflicted with the 
demonstrated benefits of using e-Textbooks in education. E-Textbooks were beneficial in K-12 education, but 
instructors in higher education were not interested in them. 

Preference to printed e-Textbooks. Even there is a choice on the formats of textbook, 33.9% instructors 
still preferred to the printed textbooks, and took printed textbooks as a premier option for teaching. Although 
there are so many features of e-Textbooks in teaching as demonstrated in the literature review such as customize 
teaching and learning, more interactive activities, and flexible learning experience, instructors “still like to write 
on the pages of books”. As an instructor from Kentucky stated that “I have no problem with this (using e- 
Textbooks), (just) thought I prefer a print copy for my use”. E-Textbooks still has a long way to go to replace 
printed textbooks. 

Disregard the formats of textbooks. Most of the instructors did not “specify printed or e-textbook”, 
their primary concern is whether the innovation is suitable and acceptable for students’ learning, not the 
textbooks’ fonnat. Instructors were open to e-Textbooks as an option, but they “would want (their) students to 
have the choice of using whichever version they prefer” and “not exclude those who may not want to read on 
their computer”. Instructors’ selections of textbooks will vary “based on learning outcomes”, they allow 
students to use texts in any form available at the least expense to them”. 

No textbooks at all. There were a number of instructors who were not fans of textbooks at all; they did 
not assign any textbooks for students. A few instructors in Alabama and Kentucky reported that, “I do not use 
textbooks and prefer research articles”, “I don't use textbooks; I do not use publisher created textbooks”, “I use 
few textbooks of any kind”. Instructors turned to open education resources in recent years, which made them 
“no longer in good conscience ask students to buy textbooks”. 

Students’ Learning Needs with E-Textbooks 

The nature of the social system where the innovation adoption happens can influence the adoption rate 
of an innovation, which includes the social norms, the degree of network interconnectedness, etc. (Rogers, 

2003). In this study, it referred to the classroom where instructors delivered their instructional activities, either 
the traditional classrooms or the online learning environment. Instructors always take students’ learning needs 
into consideration before adopting any innovative educational tool for teaching. Instructors “just require the 
appropriate book for the class. It doesn't matter if a student uses an e-textbook or hard copy,” and “provide them 
( e-Textbooks ) as an option for my students, but not requiring them. . . students should use the fonnat they are 
most comfortable,” as commented by instructors from Alabama and Tennessee. 

Students ’preference to printed textbooks. Findings of this study showed that “students appear to prefer 
printed books”, they choose “to use printed textbooks”, which made it difficult for instructors to insist using e- 
Textbooks in teaching. This finding was opposed to Kissinger’s study on e-Textbooks library (201 1) which 
found a higher interest of using e-Textbooks among students than instructors and blamed the lower rate of 
adoption e-Textbooks on instructors. However, in this study, “The use of e-textbooks is up to the students and so 
far they haven't chosen to use them”, as reported by instructors from the four states, “students are not as 
receptive to ebooks and request hard copy books”. 

Students ' inadequate knowledge of e-Textbooks. Findings from this study indicated that applying e- 
Textbooks to the classes whose students had no or little knowledge of computers was a poor idea. An instructor 
from Mississippi stated, “students are non-traditional (older) students that are not as technologically savvy as I 
would like (different from the younger, traditional college -aged students) and do not always have reliable access 
to a computer, and therefore I do not incorporate digitalized textbooks very often in my courses.” So, even if e- 
Textbooks had hundreds of relative advantages over the printed textbooks, the less compatibility of them with 
students’ previous knowledge and experience made the adoption process more difficult to move forward. 
Availability of E-Textbooks via E-Textbooks’ Publishers 
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Change agents’ efforts in promoting adoption of an innovation can influence the innovation adoption 
process; the more efforts the change agents make, the quicker the adoption would be (Rogers, 2003). Instructors 
from the four states reflected that there was a lack of the available e-Textbooks in educational related programs, 
which prevented them from adopting e-Textbooks in teaching. 

Unavailability of e-Textbooks. Findings from this study showed that one of the reasons of not using e- 
Textbooks is the lack of available e-Textbooks in their subjects. Although there were quite a number of e- 
Textbooks available, “not all texts (they) need are available as ebooks”. As instructors from Alabama and 
Kentucky stated that “I don't know of any (e-textbooks available) for science education”, and “most of the books 
we use do not have e-Textbook options”. In this case, it is impossible for instructors to choose an e-Textbook 
that is not in the market, no matter whether instructors have the intentions to adopt e-Textbooks or not. 

Quality of e-Textbooks was unsatisfied. The quality of e-Textbooks is another challenge of applying e- 
Textbooks to teaching. E-Textbooks publishers did not integrate a universal standard for e-Textbooks in all 
subjects, as they did to the printed textbooks, which made instructors difficult in locating high quality e- 
Textbooks in their fields. Instructor from Tennessee commented that “The ones available for my courses are 
weak (or there aren't any available) in content. Plus ebooks are poor for science— tables, graphs are often 
difficult to read and one cannot easily compare two pages, often necessary for dense material”. Another factor 
that influenced the quality of e-Textbooks is “print is too small on the screen”. In addition, the quality of 
illustration is not pleasant for the graphic based materials. Therefore, some instructors felt that "it is difficult to 
find a methods book in social science education that provides quality examples”. Finally, not all textbooks have 
both printed and electronic versions, so it is impossible for those instructor to offer e-Textbooks as an option. 
The e-Textbooks publishers did not have a complete promotion of the e-Textbooks in all subjects. 

Limited lifetime of e-Textbooks. Currently, e-Textbooks usually would be available for 6 months after 
purchase. As instructors from Kentucky mentioned that “several of the titles I have used are only available for 6 
months, thus the learner cannot have access after the time is up and are not available after the end of the 
semester.” The limited life of e-Textbooks prevented the students who want to keep it with a longer time as they 
can do with the hard copied from using one. If e-Textbooks publishers could open up a new or more practical 
methods for users to keep e-Textbooks longer, this might increase the use of e-Textbooks in education. 

Conclusion 

Based upon instructors’ responses regarding to using e-Textbooks in teaching, the adoption of e- 
Textbooks in higher education is related closely with the institutions, instructors, students, and e-Textbooks 
publishers. The four factors, institutions’ influence on adoption of e-Textbooks, instructors’ attitudes towards 
using e-Textbooks, students’ learning needs with e-Textbooks, and availability of e-Textbooks via publishers, 
can interact each other and influence instructors’ adoption of e-Textbooks in their teaching. Institutions have 
responsibilities to assist instructors to adopt and spread any innovation in education, technically and 
economically. Those explored factors that prevented instructors from using e-Textbooks in teaching indicated 
that to increase using e-Textbooks in teaching in higher education is not only dependent on instructors’ interest 
and their teaching responsibility, but also dependent on students learning style, e-Textbooks publishers, teaching 
and learning goals, support from social systems, etc. Keeping those factors in mind will help and improve the 
adoption of e-Textbooks in teaching in higher education. Without support from the entire social system, which 
can be the whole society, a school, an organization, or community, the adoption of any innovation is truly an 
individual action and impossible to move forward. Instructors need to examine how these can help balance the 
technology and course content when using e-Textbooks, and provide sufficient support for students’ using e- 
Textbooks in learning. E-Textbooks publishers should not only contribute to exploit features of e-Textbooks, 
but also consider a proper method to extend the life of e-Textbooks for users. 
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Abstract 

The public educational process (primary and secondary) in the state of Texas has always been subject to 
examination, influence, and manipulation by various people, governmental agencies, political parties and 
movements. These various entities have dictated, either implicitly or explicitly, the content and conveyance in the 
public educational process. Since 2010, a movement known as the Tea Party has explicitly sought to influence, 
manipulate and control the public educational process in Texas public school districts’ educational (content) and 
instructional technology (conveyance). Its influence is yet unclear, but an ever-growing presence. 

Introduction and Background 

The Texas Tea Party and its allied associates emerged as a potential influence in determining how 
educational content and conveyance (instructional technology) would be shaped and influenced in the public 
educational process in Texas. The distinction concerning the Tea Party and its influence on public education are 
twofold: ideology and intentness. Since its emergence as a political party in American society, more and more is the 
emphasis on ideological correctness, political intent in influencing aspects of human life and accountability from 
government policies and practices. This study provides an attempt to understand these intentions from the Tea Party, 
particularly how it’s ideological and intentional concepts would influence educational content and instructional 
technological conveyance in public education. 

Since the implementation of the American public educational process in the mid- late nineteenth century, 
outside forces (social, cultural, political) have vied to influence and guide the educational content and conveyance in 
the public primary and secondary educational situations. The names have changed, but the intent has not; groups 
have challenged the public education model in this country for various reasons and purposes. Now, according to 
Simon (2014), in Texas, a new permutation in the long-standing discussion over control and direction of public 
education in Texas emerges in the appearance and development of the Tea Party and its potential influence. In 
addition, certain nonprofit foundations, heavily financed and with narrowly focused agendas, are working or are 
engaged in privatizing public education (Barkan, 2013; Ravitch, 2010; Waters, Barbour and Menchaca, 2014). 

Two identifiable poles of educational theoretical influence appear to attract and motivate these outside 
forces in their relationship with public education in this country: progressive educational theory and its proponents; 
the other essentialist educational theory and its proponents (Kessinger, 2011; Schutz, 2011). Since the beginning of 
the twentieth century, these two poles of influence have developed many permutations of influence that have 
motivated proponents, activists, theorists, reform movements, and political actions to determine who will influence 
the primary and secondary public educational realm, from what Schutz (2011) argued is “progressive educators 
developing pedagogies designed to nurture the individual voice with egalitarian classrooms” (p. 1) to Kessinger 
(2011), who listed the seven major initiatives coming out of essentialist educational theory, that is the National 
Defense Education Act of 1958, Elementary and Secondary Education Act of 1965, The National Assessment of 
Educational Progress, A Nation at Risk, America 2000, Goals 2000 and the Elementary and Secondary Education 
Act of 2001 otherwise known as “No Child Left Behind.” 

Since 2010, when the Tea Party movement achieved political prominence by helping to elect a Republican 
majority to the United States House of Representatives, Republican majorities to state legislatures, and other elected 
positions, it has represented the latest permutation of progressive versus essentialist theory in influencing public 
educational content and conveyance (Dochuk, 2012; Kennedy, 2014). In Texas, since 2010, this political entity has 
been building up its control and influence of this situation on the state and local level; that pressure can be witnessed 
on local school districts in the North Texas area (Floyd, 2013; Selk, 2013; Smith, 2014). Since who controls such 
situations eventually controls how educational content and the way it is delivered to students — educational content 
and instructional technology — this study seeks to understand and analyze how such influence affects educational 
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content and conveyance and to serve as a guide in filling in the gaps left by scholarly research on such influence. 
Political forces are an element that influences the policies and practices of educational content and conveyance, and 
public educators (primary, secondary, higher education) cannot ignore this developing phenomenon (Alinsky, 1972; 
Chambers, 2003; Schultz, 2011). 

Conceptual Framework 

The political situation in Texas and other states where the Tea Party exerts some kind of influence is 
another permutation of the long-term conflict over educational policy and practice between proponents of 
progressive educational theory and essentialist theory. E.D. Hirsch (2009) stated that progressives, or as he termed 
them, “the anti-curriculum movement,” have contributed to the stereotyping of essentialists by the use of “effective, 
polarizing slogans that have been erected over the years, a propaganda effort that continues to preserve the anti- 
curriculum doctrine in the face of failure and decline” (p. 53). Other scholars concur with Hirsch on his anger for the 
obstructionist ideas of progressive educational theorists and that public education cannot be continued in the old way 
as before in its practice (Kessinger, 2011; and to some degree, Peterson, 2003). Peterson (2003) pointed that the 
seminal icon of defining the inadequacy of the public educational system, the National Commission on Excellence 
in Education’s famed 1983 study America at Risk, highlighted what Ronald Reagan’s secretary of education at that 
time — Terrel Bell — announced to the nation as a report on how the country’s educational institutions lost sight of 
the basic purposes of education (from primary and secondary to higher education and the disciplined effort needed 
to attain them (p. xvii). 

In contrast, progressive educators feel that the insistence on “accountability,” one of the primary goals of 
essentialist educational theorists, is really a mask for destroying public education and one of its main principals, that 
of teaching egalitarianism. Certain scholars (Giroux, 2012; Price, Duffy & Giordani, 2013) strongly defended public 
education from what they call the venture capitalist class. Price (2013) made clear that he and his colleagues are 
publishing and have joined with students, parents, teachers, and union leaders in the Midwest to fight on behalf of 
public education. Joining with them is Diane Ravitch, a former official in the Department of Education during the 
George W. Bush presidency, who first believed that the proponents of this essentialism, or the new term of 
“accountability,” put forward a good idea. As she put it, Ravitch (2010) argued that “in the decade following my 
stint in the federal government, I argued that certain managerial and structural changes — that is choice, charters, 
merit pay, and accountability — would help to reform our schools” (p. 8). With such changes, teachers and schools 
would be judged by their performance; this was a basic principle in the business world. Towards the end of the first 
decade of the twenty-first century, Ravitch (2010) experienced a change of heart; she noted that while sounding 
appealing, the one fatal flaw in this thinking was that a person could supposedly ignore anything about education or 
children. Ravitch further elaborated that “the lure of the market is the idea that freedom from government regulation 
is a solution all by itself. This is very appealing, especially when so many seemingly well-planned school reforms 
have failed to deliver on their promise”(p. 11). In a sharp contrast to her earlier support, Ravitch stated the new 
corporate reformers exhibited a strong ignorance of what education is all about when they attempt to adapt it to a 
business-type model (p. 11). Giroux (2013) echoed support for her analysis when he stated that when not functioning 
as a business or a lucrative for-profit investment, public schools have become containment centers. The whole point 
of the contrast between essentialist educational theory and progressive educational theory is that they are being used 
to assert control over the public educational process for certain gains and are the context in which a contest for 
influencing public education is happening (Hirsch, 2009; Giroux, 2013). Today, that conflict continues in the state 
of Texas and manifests itself in Texas politics and education. To some individuals, this political movement 
represents an opportunity to reform and improve the public educational system in the United States and to some 
individuals, it represents another power grab for the powerful to exploit and control what people think. If this 
situation is the context in which primary and secondary educators, administrators, associated organizations and 
activists must function, then it is necessary to perceive how such a context affects them and can be affected by them. 

Aaron Schutz (2011) provided a starting point when he talked about how many educators felt today about 
the public educational process; he noted that many progressive educators still hope that the idea of ‘the school as a 
democratic community ’(p.l) can still be a means to reforming society around the principles of social justice and 
democratic equality. In his discussion he pointed to a political power theory that explains how decisions are made in 
an actual societal setting, based on the ideas of Saul Alinsky, as expressed in his writings and the development of 
his institution — the Industrial Areas Foundation through the coordination of two of his disciples, Richard Harmon 
and Edward Chambers, who codirected and coworked with Alinsky. Harmon and Chambers developed their ideas 
about the political process on American society and refined the training and organizing methods in this institution 
that were needed in order to be an effective advocate for their political ideas. 
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The point here is that professional educators perform their task in an environment replete with potentially 
influential forces, chiefly political, and must recognize that condition. They cannot conduct or pretend to do research 
or practice under the assumption of the education situation being an insular environment; the forces examined here 
are political forces with unknown and unclear, yet potential and influential consequences in their educational 
situation. Through the template provided by the ideas of Saul Alinsky, the principles developed by Richard Harmon 
and Edward Chambers, and implemented by activists such as Ernesto Cortes, it is hoped that an initial awareness of 
such an influence, without judging its potential, can be culled from discussion with the literature, educational 
professionals (politicians, administrators, teachers) and associated personnel and entities. 

Tea Party Political Activity and Influence in Texas 

In the state of Texas, the public educational process has always been impacted by political and social 
influences in terms of what and how it is done, be it from whatever political party in charge. What has changed since 
2010, (Floyd, 2013; Kennedy, 2014) out of the Republican ascendancy in political power in Texas, is the ideological 
and intentness of change on a large scale. The Texas Tea Party achieved noted success in November, 2010, when it 
helped the Republican Party solidify its lock on the political process in the state. The Texas Tea Party movement has 
moved the Republican Party platform to the right on such issues as abortion, politics, and education (Floyd, 2013; 
Kennedy, 2014). Through their adept use of the Internet, these like-minded individuals have stayed in 
communication, worked with like-minded organizations such as Empower Texas and the Texas Public Policy 
Foundation to exert a never-ending influence on politicians on different issues (Kennedy, 2014). The influence of 
the Tea Party and its ilk-like associate entities is not just a blossoming and withering fad. It is a political process in 
the making for decades. Eventually, according to experts, this type of organization would find a home in the 
Republican Party. David Dochuk (2012) noted that such a process has been underway for years with people 
organizing around key issues such as abortion and prayer in public schools. What has been helpful to the Tea Party 
in achieving its breakout year in 2010 is the emergence of key personalities who provide the pedagogy, 
philosophical and educational structure for Tea Party adherents. According to Dochuk (2012), a key player is David 
Barton, an Oral Roberts University graduate and former co -chair of the Texas Republican Party, who has actively 
worked to increase the Tea Party and conservative religious influence in Texas state politics and served as chief 
advisor for the National Council on Biblical Curriculum in Public Schools (encompassing 555 schools in thirty-eight 
states, attended by 360,000 students), and led a campaign to change the Texas State Board of Education’s history 
curriculum. Bud Kennedy (2014) noted the influence of such leadership on the dominant Republican Party in Texas 
Politics; an example of this influence is on the issue of illegal immigration. Whereas once President George W. 

Bush said that “the vast majority of illegal immigrants are decent people who work hard, support their families, 
practice their faith and lead responsible lives” (Kennedy, p.l), now the party’s movement conservatives have turned 
Republicans toward not only towards tougher enforcement but also on shooing away immigrants away simply 
because they’re not as conservative. According to Cal Jillson, Professor of Political Science at Southern Methodist 
University in Dallas, Texas, the Tea Party was able to lay the blame for problems with President Barack Obama’s 
election in 2010 and 2014 and the coincidental hard economic times of that period, which helped the ascendance of 
the Tea Party over the long-dominant business wing in Texas (Kennedy, 2012). 

Early Attempts of Control and Tea Party Attempts to Control 

One of the differences in this motivated effort and previous motivated efforts can be a comparison in the 
actions of people who believed that the educational process in Texas was not in favor of traditional values and ideas. 
Floyd (2013) cited the example of a couple, Melvin and Norma Gabler, who lived in Texas and who would regularly 
travel to Austin to testify before the Texas State School Board about their complaint on certain content in school 
books, chiefly the lack of discussion of moral and traditional values. Compared to such activities back in the 1980’s, 
she observes how William Martin, author, social scientist and fellow at the Rice University’s Baker Institute for 
Public Policy notes today ‘there is a tremendous amount of cynical manipulation going on’ (p.6) by politicians using 
conservative religious ideology to shore up their power bases compared to such events thirty years ago. Floyd 
(2013) continued by quoting Martin as saying that they’re targeting evolution, climate change, and any material they 
have arbitrarily labeled as ‘biased’ or ‘unpatriotic’(p.6) and he feels less sanguine about the motivation behind 
current witch hunts for ‘liberal bias’ in public school teaching materials. To Floyd (2013), the heightened 
manipulation is easily displayed in the 2013 Texas Legislative session where a few state legislators displayed their 
talent in whipping up an ideological dervish dance over a state-developed computer-based curriculum package 
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known as CSCOPE, a situation discussed a little later. Starting in 2013, the Tea Party has attempted various efforts 
to influence and control educational content and conveyance. 

Tea Party Attempts to Control the Educational Process 

In 2013, people aligned with the Tea Party movement and philosophy, flexed their ideological and 
intentional influence over what was known as the Service Center Cooperative Lesson Plan system, known as 
CSCOPE (Floyd, 2013; Selk, 2013). CSCOPE (Under the Dome, 2013) was an electronic curriculum management 
system that offered Web-based lesson plans and exams for teachers to use in lesson plans and potential tests in their 
classroom situations and it was developed by educators over a period of years at regional resource centers 
throughout Texas, one of the purposes being “to help teachers adhere to state educational requirements” (p.2) It was 
a popular program used in 857 school districts — 70% of the school districts in the state of Texas. However, parents 
and Tea Party activists actively campaigned against it in 2013, thus eliminating its use in the state educational 
system. Stutz (2013) observed an example of how the Tea Party influenced educational decisions in the state of 
Texas occurred when the chairman of the Senate Educational Committee, Dan Patrick, in a special session of the 
Texas Legislature in July, 2013, prepared to file a bill “to prohibit use of a controversial series of lesson plans in 
Texas schools in Texas schools — even though hundreds of districts and charter schools are prepared to use them in 
the coming school year” (p.2). Stutz (2013) noted that Senator Patrick’s effort was prompted by the fact that the 
Texas Educational Agency said such material is in the public domain and can be used by any school district. 

This school district went from being a strong user of the CSCOPE material to forcing educational 
administrators and educators to abandon it and develop an alternative curriculum two weeks before school started. 
Selk (2013) mentioned that up until May 2013, this North Texas school district had integrated CSCOPE into its 
educational year; he noted that there had been some hiccups such as bogging teachers down in paperwork and 
inflexibility for teachers to use optional material, but the school district leadership (pre-Tea Party) felt comfortable 
with the potential benefits. To Tea Party activists and social conservatives involved in the political process in this 
school district, the biggest problem was what was perceived to be a chief objective of using CSCOPE — that of 
teaching and forcing the acceptance of Islam and socialism. Selk (2013) pointed out that tea party oriented school 
board trustees loudly complained that CSCOPE lesson plans indoctrinated students into Islam and socialism. At the 
last minute, in the 2013-2014 school year, this Texas school district had its educational content and conveyance 
altered significantly by political influences in the district 

Influence of Progressivism Versus Essentialism 

Schutz (2011) and Kessinger (2011) offer insight into discussing the motives and theories behind these 
political struggles over controlling education. Schutz (2011) keenly postulated that progressivism, primarily under 
the influence of John Dewey, looked on in horror at the constant conflict between labor and management; they felt 
that the unions were too focused on conflict and mass solidarity and the rich as driven by too much greed and lack of 
social compassion. Schutz (2011) noted how progressives focused on students learning social skills such as learning 
to cooperate with each other on tasks and engaging in group thought to solve problems. Even today, according to 
Larabee, it is noted that even in the era of “basic skills achievement era of No Child Left Behind” (p. 142), most 
educators remain intellectually and emotionally committed to the one of the main ideas of progressive educational 
theory, that being “the school as a model democratic community,” using the idea of educational reform to reform 
society around the principles of social justice and democratic equality. 

In contrast to progressive educational theory, essentialist educational theory focuses on developing a 
competent and skilled person, with focus on teaching reading, writing, and arithmetic, liberal arts and science, and 
other academic disciplines (Hirsch, 2009; Kessinger, 2011). The essence of essentialism is dealing with factual 
knowledge. In this fight between progressivism and essentialism, whatever the permutation, particularly since the 
mid-twentieth century, the fight comes down to recreating the democratic process, influenced by the thoughts and 
ideas of John Dewey, versus accentuating the basics, that is content, skills, and disciplines found in the content of 
education (Giroux, 2012; Kessinger, 2011; Price, Duffy & Giordani, 2013; Schutz; 2011). 

Research Problem and Questions 

Out of this context, the research problem emerged with the intent of discerning the impact of new political 
organizations such as the Tea Party. Studying the Tea Party political activity and influence on public education helps 
to shape educational content and develop instructional technology practices that possibly improve public education. 
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Two research questions guided this study. Research Question One is the perceived implications of the Tea Party 
movement on Texas school district educational process and educational and instructional technology in the 2013- 
2014 academic year and the just completed 2014-2015 academic year. Research Question Two focused on what are 
the perceptions of professional education administrators, educators, educational-related entities in the North Texas 
area about the Tea Party movement and its potential influence on their educational and instructional technological 
practices. 


Method 

In order to gauge the potential impact of the Tea Party activities in public education, a purposeful sampling 
of educational professionals, associated entity organizational staff, university scholars, and political observers and 
activists involved in the public educational process (primary and secondary) in a particular North Texas school 
district and surrounding school districts will be studied using a phenomenological qualitative case study research 
design in order to gauge their perceptions about the potential influence of the Tea Party on educational content and 
instructional technology conveyance in public primary and secondary schools in Texas. A sample size of at least ten 
educational professionals being interviewed will focus on the perception of this situation as a positive or negative 
social change. The site of this study is the North Texas area of the state of Texas, chiefly the cities of Dallas and Fort 
Worth, Texas and surrounding suburbs. 

I conducted a phenomenological qualitative case study research analysis of this political influence on 
educational content and instructional technological (conveyance) development in regional educational entities, 
starting with the above mentioned school district, noted for Tea Party control and influence on its school board 
elections in 2012. In the school board elections of that year, Tea Party backed candidates won a majority of the seats 
on the school board and began implementing content (policy) and instructional technological (conveyance) policy 
changes. I sought to talk with people on both sides of the issue. In an attempt to be fair-minded about including all 
perceptions about the growing influence of the Tea Party in Texas education, I attempted to reach out to individuals 
in this district who might represent a different point of view. My attempts were met with suspicion and silence. 

Results 

The results gathered from this study were grouped into how they fit into the two research questions for this 
situation. In regards to the question about the perceived implications of the Tea Party movement on Texas school 
district educational process and educational and instructional technology in the 2013-2014 academic year and the 
just completed 2014-2015 academic year, one theme that emerged was the realization that there exists an ideological 
and intentionally sustained effort to influence the public education of children and youth in the United States. The 
Tea Party phenomenon is one among many political creations designed for such efforts and deserves an attempted, 
determined focus on its developing existence. 

For example, in the phenomenological study findings of this study, in interviews with three activists, 
familiar with organizing, observing, and making political decisions affecting public education in Texas, the theme of 
recognizing political influences on public education and associated human activities cannot be ignored. One of the 
individuals, active with a major social organization, working on social issues in the North Texas area, when asked 
the question, “Why is it important to understand how the political process works involving an activity like public 
education?,” stated that in his work of focusing political efforts among people and organizations, as a local 
community organizer, one of the things that he noted is that people need to understand the hopes and hazards of 
living in a democratic society. He pointed out that politics and its effects on different aspects of human existence is a 
given and cannot be ignored. In an activity such as public education, political control and its influence cannot be 
ignored. 

In relation to the Tea Party’s political activity and influence in Texas public education, another individual 
involved on the State Board of Education, emphatically stated that the Tea Party began their attempt to politically 
control the State Board of Education before 2010. In his opinion, what you have is a bunch of white, upper middle 
class, Republicans who don't want to send their children and grandchildren to school with a bunch of poor minority 
kids. So, the answer, according to this expert, is to denigrate the public schools' reputation, advocate for "letting the 
money follow the child" and support charter schools. Another observer, working in a political organization based in 
Austin, Texas, when asked the above question, in his own opinion, discussed how many political groups are working 
to denigrate and limit resources to public education in order to divert resources to their favored projects. 

In looking at the first research question, another theme that emerged from the interview process with these 
individuals was that of ideological bent. In interviewing two individuals associated with governance of school 


186 



districts, when asked the question “Do you see a political environment right now in Texas politics that is influencing 
or changing how your professional classroom teachers do educational and instructional technology (content and 
conveyance) delivery?,” these school board members remarked and elaborated the following observations on the 
Tea Party’s potential influence. According to one interviewee, once Tea Party-backed trustees ascended into 
domination in this North Texas school board, the use of the teacher developed CSCOPE curriculum was 
immediately terminated, the bilingual education system was eliminated as much as possible, with the emphasis on 
total immersion in English. According to this person, the reason that the bilingual program was not gutted more was 
because of Federal and State regulations. The practice of dual language acquisition skills was frowned upon, even 
though the ethnic makeup of the school district was 76% Hispanic, 20% White, and 7% Black. According to this 
person, this biggest impact of the Tea Party on instructional technology is the ideological influenced lack of interest 
in discussing, planning and implementing a school bond program for infrastructural needs. In the state of Texas, 
school bond elections are used for upgrading a school district’s infrastructural needs such as buildings, either 
directly or indirectly associated with the district’s educational mission. In addition, items related to technology, 
included in a school bond election, are technologically-enhancing necessities for upgrading Internet speed and 
capability for various computer programming and technology. 

Another individual involved in the governance of a large urban school district, when asked the same 
question, responded by saying that while no intentional purpose was detected yet in that school district, an 
intentional and ideological effort to control public education content and conveyance was developing in Texas, and 
the Tea Party was part of that effort. The individual pointed to a movement that was financed to turn the large urban 
school district into a local charter situation that could instigate rules and regulations partial to Tea Party ideologues. 
According to this individual, in agreement with the educational expert, the purpose of public education is no longer 
about education; it is the new Jim Crow law designed to slight and stymie minority students’ education. Continuing 
in the discussion of the theme of the importance of ideology, one of the findings of the phenomenological study was 
this pattern of noninterest and hesitancy to fund, manage, promote, and plan for the future for public education. This 
view is seconded by a local leader of a teacher’s union. When asked the question, what do you see as the current 
environment in Texas education right now? In terms of resources being allocated to primary and secondary 
education”? This individual noted that tea party affiliated politicians and activists viewed the public school system 
in Texas as being influenced by “secular humanism and opposed to supporting “traditional values.” 

Another finding in the phenomenological study, in relation to the theme of ideology, noted by a history 
professor familiar with Texas politics, was the discussion about the attitude of Tea Party members towards public 
education. In response to the question, “how do you perceived the Tea Party and its ideology shaping and 
influencing the way government policy and procedure is done in the next few years, particularly as it relates to 
educational policies and practices?” his response was that the Texas legislature would be loath to fund public 
education to adequate levels envisioned by educational professionals because Tea Party sympathizers who believe 
governmental agencies need to be small. He further added that Tea Party believers and their influenced politicians 
would be interested in such public education funding approaches such as vouchers, charter schools (both online and 
onground), private and religious schools. He emphatically stated that Tea Party ideologues and sympathizers did not 
trust the public education process. 

The second question is the perceptions of professional education administrators, educators, educational- 
related entities in the North Texas area about the Tea Party movement and its potential influence on their 
educational and instructional technological practices. In the interview process with the above mentioned individuals, 
some interesting themes appeared. First, according to the individuals interviewed above, politics is going to 
influence educational content and conveyance. As noted in the contextual framework section, once the tea party 
organization gained political control of the above mentioned school district, certain content and instructional 
conveyances were changed to suit the wishes of those Tea Party believing school board members. Educational 
professionals view the political influence situation with many types of attitudes and approaches. 

In assuming that political influence is part of the educational process in a public environment, the three 
approaches encountered in the gathering of information for this study was either one of passive acceptance or active 
resistance or active accommodation. In the passive acceptance mode, the educator accepted the political situation as 
reality and made adjustments accordingly; in my approach to gathering information for this study, many educational 
professionals were loath to discuss or express their views on the issue, even when guaranteed anonymity. In the 
active resistance approach, the individual would work through their local teacher, administrator, or school board 
association to thwart perceived Tea Party agendas. Often local teachers’ unions are very active in urban school 
districts’ school board trustee election situation in order to protect their agenda. Finally, a third approach discovered 
was trying to actively engage the developing situation and seek to work, neutralize, or co-opt it. This third approach 
proved very interesting to examine as a reaction to political influence and activity. 
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An example of this third approach to political awareness occurred when I interviewed a top-ranking 
administrator for another local school district about the influence of the Tea Party on educational content and 
conveyance in that individual’s school district. When she was asked the question, “what do you see as the current 
environment in Texas education right now?,” she stated that the top administrators in her school district saw the 
changing educational environment and chose to be proactive and work with the newly developing environment 
instead of reacting to it. One of the things that her school district did was set up agreements with local charter 
schools for transferring of credit and create specialized-focused academies for students to attend, such as math 
academies. That administrator felt that by being aware and proactive, the school district was able to minimize such 
political influence and activity’s potential dismption and work to enhance its educational purpose. 

Discussion and Implications 

One of the insights that individuals associated with the public education process in the United States must 
recognize is the fact that politics are a part of that process and initiative. Chamber (2003) and Schutz (2011) noted 
the distinction that must be made by progressive educators in the world of human experience, of which education is 
part and parcel, and that is understanding the tension of the world in terms of how it functions in human existence. 
Chambers (2003) observes that in the world as it should be, democracy means participation in public decisions in 
which all are included because of the dignity of being “created equal” (p. 14) In the real world, democracy is 
dominated by the interests of a few wealthy and powerful institutions. He notes that one must accept two conditions 
of the world in which human beings exist; a world of needs and necessities, opportunities and limitations, and what 
he calls the “real world” (p.14). In contrast to that situation, Chambers (2003) pointed out that human beings have 
dreams and expectations, yearnings and values, a sense that there is an ideal, a greater good that matters, and if we 
do not attempt to reach that ideal, then we know that we have not fulfilled ourselves. 

Chambers (2003), Harmon (2011), Schutz (2011) insisted academics must realize that educational and 
instructional technology does not exist in a vacuum. Too much information on the educational process focuses 
inwardly on the process and angles associated with that process. Annette Lareau (2003) and Patricia Hill Collins 
(1990) insisted that this lack of awareness came from the middle-class practices of children of professionals who were 
asked for their opinions, participate in dialogues about issues, lived in a world of negotiations, existed with uncertainty, 
not established traditions and mores. Within this context, it is important to pay attention to these outside influences, 
particularly in light of its interrupting the educational goals, content, and conveyance of this school district. The 
importance of realizing that the core belief of keeping a democracy is where ordinary citizens organize and cultivate 
the virtues and skills of democratic citizenship (Stout, 2010), and in such a key professional task as they engage in, 
professional educators cannot dismiss or assume it does not affect them (Lareau, 2003). 

Schutz (2011) points out how Harmon, Chambers, and other political organizers pick apart the weaknesses 
of middle-class progressives (particularly educational professionals) by pointing out that they intrinsically operate 
on a fatal flaw Dewey taught in his Laboratory school — that people need to trust one another in the democratic 
experience. People, particularly professional middle-class individuals, according to Harmon, (Shutz, 2011) attempt 
to avoid confrontation in discussing and addressing divisive issues. Alinsky (1971) noted that the basic requirement 
for understanding a theory of political or power change is that one must recognize the world as it is. Alinsky (1971) 
further elaborated that most people view the world, particularly the professional middle-class individual, with self- 
developed illusions, not the way it is but the way they would like it to be (Collins, 1990). Such a self-induced 
delusion must be faced realistically. 

Selk (2014) reported that one of the actions taken by the new superintendent, was announcing that nearly 
$750,000 worth of software packages, contracts, tutoring programs, and other items would not return in the 2014- 
2105 school year; the superintendent stated that “The departments that worked with these things are the ones that 
submitted the preference to abandon them.” (p.5) The former school board member that I interviewed noted that 
some of these content and conveyance material were ended because of Tea Party school board members’ perception 
of representing multiculturalism and other un-American beliefs. If such disruptions are part of a political situation, 
then that political development needs study. 


Conclusion 

Educational content and conveyance does not operate in an insular environment; sometimes it appears that 
studies on educational content and conveyance convey operating in a perfect atmosphere. Such insular studies need 
to get realistic. Human beings operate in a messy and multi-influential environment; that stated environment and 
situation needs to be addressed and examined. As Waters, Harbour and Menchaca (2014) noted, one of the gaps in 
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analyzing charter schools (online and onground), one of the tools favored by school reformers and Tea Party 
influenced individuals in Texas, is that more research needs to be done on the complex implications associated with 
such alternative educational practices and accompanying issues. 
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Abstract 

This study explores and evaluates the ways a university engaged in systemic change through the integration of 
faculty development and the development and implementation of two academic technologies designed. Both the 
faculty development and the technology tools were used to encourage the creation of program and course 
learning outcomes. The results show that the collaboration of instructional designers, faculty consultants, 
technology developers and faculty have enhanced the design, development, and support of these academic 
technology tools and have brought about a large-scale use of learning outcomes in program and course designs. 

Introduction 

The purpose of this study was to document and evaluate systemic change at a large, mid-western university 
where faculty were being asked to create program and course learning outcomes and collect data on student learning 
related to those outcomes. The university used faculty develop and two academic technologies to support the 
implementation of learning outcomes in programs and courses. The two academic technology tools were a learning 
outcomes website and a learning management system. The faculty development focused on helping faculty see that 
good course design follows the same pattern as good research. 

Faculty often are not as familiar with course development as they are with research methodology so the 
faculty development focused on helping faculty see that the methodology they use in research design is similar to 
the way they should approach course design. The emphasis of the faculty development on learning outcomes was 
integrated with the development of a learning outcomes website and a learning management system. Learning 
outcomes created for programs and courses are linked to the syllabus in the learning management system. Faculty 
were shown how they can link assignments and class schedules to learning outcomes so students see how learning 
outcomes support class instruction. 

Data was collected on the faculty use of the learning outcomes website and the learning management 
system. Faculty were interviewed to see how they were implementing the connection between learning outcomes 
and their course development. Results indicate that the integration of faculty development based on course design 
focusing on learning outcomes and the design and implementation two academic technologies that facilitated the use 
of learning outcomes in course design brought about system change at the university. Faculty began to see the 
importance of course design focused on program and course learning outcomes. 

Review of Literature 

A current trend in higher education is focused on improving student learning by changing instruction from 
a teaching focus to a learning focus (Barr & Tagg, 1995; Karseth, 2006; TRENDS, 2010). In fact university and 
schools of education are facing accreditation standards that require them to create program and course learning 
outcomes, and to collect evidence that demonstrates that students have met the defined learning outcomes 
(Association of American Colleges and Universities, 2002; Northwest Commission on Colleges and Universities, 
2010). In their research on faculty development Lightner and Benander (2010) have found that some faculty 
members have been resistant to writing program and course learning outcomes for several reasons listed here: 
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Faculty do not have time to create them 

Faculty do not see learning outcomes as consistent with their teaching philosophy 
Faculty feel that writing learning outcomes is not part of their job descriptions 

Faculty are under too many other academic pressures including the need to publish in order to be granted 
tenure 

Faculty feel that learning outcomes are just the current fad in education, just another “hoop to jump 
through'’ for accreditors 

In order for faculty to overcome these attitudes about developing learning outcomes, many colleges are 
creating Faculty Centers or Centers for Teaching and Learning to help faculty design courses that have this new 
focus. Faculty development is designed to help faculty understand that well-constructed learning outcomes are an 
essential component of course design and will help improve student learning. (Fink, 2003). Faculty development has 
been changing from how to write a syllabus and create exam question to a mentoring of faculty through the course 
design process stressing instructional design, effective instructional practices, and assessment options (Tam, 2000; 
Pankowski, 2004; Spicer & DeBlois, 2004). Universities and private companies are designing academic technology 
tools that support faculty as they develop learning outcomes and in some cases are asking faculty to participate in 
the design of academic technology tools. 




Research Methodology 

As part of the systemic change process towards a more learning focus, approximately 2000 faculty 
members at a large mid-western university have been required to develop program and course learning outcomes 
that would indicate how student learning would be assessed. For the past four years the faculty have been asked to 
report their learning outcomes, assessment data, and plans for program improvement on a university developed 
learning outcomes website, Expected Learning Outcomes (ELO) website. They have also been asked to use a 
learning management system developed by the university. Learning Suite (LS). Faculty consultants helped faculty 
see how LS could help them design courses, communicate with students, share content, give online quizzes and 
exam, provide student feedback, submit grades, and more. 

For the past four years the researcher has conducted a situated evaluation of the processes of institutional 
change associated with the technology development. Situated evaluation is a process-oriented approach and focuses 
on identifying the ways in which systemic reform and technology integration is realized in the practices of 
individuals (Bruce & Rubin, 1998). To trace the changes in both the academic technology development and the 
faculty development the researcher used case studies (Merriam, 1990). The case study data collected included 
detailed observations of the efforts of the participants related to the faculty and technology development, interviews 
with randomly selected participants, yearly electronic questionnaires from the researcher, and an analysis of final 
technologies and the ways in which they are used by faculty. 

Faculty development was conducted by a group of five instructional designers who focused the 
development on improving the design of courses. A major emphasis was on the use of learning outcomes to set the 
purpose of the course and to align instructional activities and assessment. Faculty often were not as familiar with 
course development as they were with research methodology so the faculty development helped faculty to see a 
similar methodology in research design as in course design (see Figure 1). In research the key questions are a) What 
do I want to know, b) How will I know, and c) What does the data tell me? The first question is answered in 
research design by establishing research questions, and is answered in course design by establishing learning 
outcomes. The second questions is answered in research design with methodology and data collection. In course 
design it is answered by aligning instruction and assessment to the learning outcomes. The last question in research 
is answered by reviewing results and considering the implication of the research. In course design the final question 
is answered by looking at student data that shows how they have met learning outcomes. 

At the same time that the consultants were working with faculty to understand course design, the ELO 
website and LS were being designed and implemented. The consultants helped faculty see that these new academic 
technology tools would make course design more efficient and effective, and consequently improve student 
learning. They collaborated with faculty to write outcomes that emphasize the development of students’ abilities, 
attributes, and knowledge that would give meaning and purpose to what students do in your course. They were 
especially attentive to the development and alignment of learning outcomes to quality learning activities and quality 
assessments in course development. The teaching and learning consultants specifically helped faculty identify, select 
and use features in the EOL website and LS to achieve their learning goals. Learning outcomes were shown in the 
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Schedule section of LS to link the discussion and assignments of that day to the course learning outcomes (see 
Figure 2). 


What Do I Want to Know? 


Research Questions Learning Outcomes 



Methods and Data Collection Instruction and Assessment 


What Does the DataTell Me? 


Results and Implications Course Improvement 


Figure /: Linking Research Design to Course Design 



Figure 2: Learning Outcomes Used in Learning Suite Schedule 


Another tool that faculty were encouraged to use was in the Assignments section of LS where each assignment 
could be assign to an assignment category. Faculty members were encouraged to use course learning outcomes as 
their categories so that both the faculty member and the students could see how each assignment related to a course 
or program learning outcome (see Figure 3). 
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Figure 3: Assignment Categories Aligned with Learning Outcomes 

LS supported successful course design by providing prompts for significant learning outcomes that focused 
on the student activities, student opportunities to interact with the instructor and other students, and formative 
learning assessments that would gauge how well students attain the learning outcomes. As the faculty development 
was implemented, suggestions and comments from faculty were brought back to the technology developers to 
enhance and improve the academic technology tools. 


Results 

Both faculty development strategies and the two new academic technologies, the ELO website and LS, 
have helped faculty create learning outcomes and integrate them into their course design. In 2010 only 1027 courses 
at the university were using learning outcomes but in 2014 the number of courses using learning outcomes had 
grown to over 3600. In the same four years all programs in every college developed learning outcomes. To verify 
that the faculty were using a variety of learning outcomes based on Bloom’s Revised Technology (Anderson and 
Krathwohl, 2000), the consultants along with three graduate students coded all of the program learning outcomes 
from 14 colleges on campus. The number of learning outcomes varied based on the size of each college. When a 
learning outcome coidd be coded into more than one category, the highest of the revised categories was used. The 
Bloom’s Revised Technology categories of Apply and Create were the most common categories for the learning 
outcomes in almost every college (see Figure 4). 
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Figure 4\ Learning Outcome Categories by College 

In the past four years, the use of LS has grown from under 60% of faculty to just over 85% of faculty. 
Faculty have collected data showing how students are meeting learning outcomes, and have articulated planned 
actions for programs when students are not meeting the outcomes (see Figure 5). The collaborative nature of the 
consultants, faculty, and technology developers has improved the design and implementation of good instruction and 
better academic technology tools. 
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Interpretive and Communicative Contexts 
English graduates know how to trace the 
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Course Embedded Assessment Rubric scores 
from the students' final papers in the core writii 
classes (295. 381-384. and 495) are collected 
each semester Analyzing patterns in the s 
specific rubric criteria dealing with stud qj 
understanding of literary tradition and^^Kirlv 
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Over the last four to five semesters, the 
average rubric scores for the final papers in 
295 for scholarly engagement have ranged 
from 3.6 to 4 4 The average scores of 381- 
384 papers for scholarly engagement 
ranged from 3 4 to 4 8. The average scores 
of 495 papers for scholarly engagement 
ranged from 3.2 to 4.4 
See the files attached to the Course 
Embedded Assessment measure of the first 
program learning outcome for more details. 
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Course Embedded Assessment On 2 
September 2015 the assessment committee 
will provide the course coordinators with the 
compiled rubric scores and preliminary 
analysis. The course coordinators will 
consider how to help a larger percentage of 
students achieve mastery level in the use of 
sources and contexts in their papers 
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Figure 5: Data Used to Show that Students are Meeting Learning Outcomes 
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Interview data and questionnaire responses from faculty also support the understanding that faculty have 
regarding the importance of using learning outcomes in their course designs. One faculty member responded to the 
interview question about how learning outcomes have improve student learning in his classroom by saying, “They have 
helped both the students and me see how all of the things we do are aimed towards accomplishing certain goals and 
how integrated the class is.” Another faculty membered stated, “Learning outcomes are great because they establish a 
target for the reasons why you’re teaching what you’re teaching. They help you as a faculty member formulate learning 
activities that will enable the students to meet the outcomes.” In responding to the second interview question asking, 
“How has the university’s emphasis on creating and using learning outcomes changed the way you teach, if at all?” a 
faculty member said, “I’ve included them on my syllabi in Learning Suite, so it’s part of my discussion at the beginning 
of the semester of the class about what I want to accomplish with them during that semester.” Another faculty member 
mentioned that, “The point of learning outcomes is it gets you thinking about what might be measurable. Student 
learning is always going to exceed learning outcomes. The outcomes aren’t driving the student learning, but help the 
teachers design student learning activities.” 


Discussion and Implications 

Academic technology tools are being designed to support systemic change in higher education that moves 
instruction from a teaching focus to a learning focus. This study has shown that the integration of learning outcomes 
into the functionality of the academic technologies has been essential to systemic change across campus when it is 
integrated with new faculty development. Faculty development needs to include an understanding of the value of 
learning outcomes that will improve student learning. Learning management systems should link program and course 
learning outcomes to assignments, schedules, class activities, and assessments. Faculty development should encourage 
faculty to use features in the learning management system that will help clarify the focus of a course. The collaborative 
work of the faculty with instructional designers and consultants will help faculty create a framework for coherent, 
integrated course design. Feedback from faculty who are creating well designed courses using the academic technology 
tools can be brought back to instructional designers and technology developers to improve the learning technologies 
they develop. 

The data collected for the use of LS and the ELO website as well as the observations and interviews of faculty 
showed a strong growth of the use of learning outcomes in course design. Faculty development and academic 
technology tools can support systemic change in higher education that moves instruction from a teaching focus to a 
learning focus. This study has shown that the integration of faculty development focused on course design using 
learning outcomes and the development of academic technologies using learning theories has been essential to systemic 
change across campus. 
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